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IS.  ABSTRACT 

This  thesis  describes  a  methodology  for  the  analysis  and  synthesis  of  modern  general 
purpose  file  systems.  The  two  basic  concepts  developed  are  (1)  establishment  of  a 
uniform  representation  of  a  file's  structure  in  the  form  of  virtual  memory  or  segmen¬ 
tation  and  (2)  determination  of  a  hierarchy  of  logical  transformations  within  a  file 
system.  These  concepts  are  used  together  to  form  a  strictly  hierarchical  organization 
(after  Dijkstra)  such  that  each  transformation  can  be  described  as  a  function  of  its 
lower  neighboring  transformation.  In  a  sense,  the  complex  file  system  is  built  up  by 
the  composition  of  simple  functional  transformations.  To  illustrate  the  specifics  of 
the  design  process,  a  file  system  is  synthesized  for  an  environment  including  a  multi¬ 
computer  network,  structured  file  directories,  and  removable  volumes. 
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Introduction 


ILS&lJJiiSIl  2l  HIS.  iiSiStS. 

The  evolution  of  general  purpose  file  systeas  parallels 
very  closely  the  evolution  of  operating  systeas.  This  is  net 
surprising  since  the  concept  of  file  systeas  grew  cut  cf  the 
eafcryonic  input-output  control  (IOC)  functions  of  early 
operating  systeas  and  now  represents  the  aost  significant 
conponent  of  aost  aodern  operating  systeas. 

There  has  been  very  little  attention  foraally  directed 
to  the  specific  problea  of  analyzing  operating  systeas.  In 
1967.  Saul  Rosen  collected  together  aaterial  fer  a  beck. 
•'Prograaaing  Systeas  and  Languages"<Rosen  67>,  which  was  to 
be  a  distinctive  selection  of  previously  published  and 
unpublished  reports  describing  the  aost  iapertant 
prograaaing  languages  and  discussing  aany  of  the  test 
iaportant  operating  systea  concepts.  He  was  forced  to 
conclude: 

NThe  paper  on  Operating  Systeas  was  prepared  for 
presentation  at  the  University  of  Hicbigar 
Engineering  Suaaer  Conference,  dune  19-29,  1962. 

It  has  had  fairly  vide  circulation  as  Band  Qepcrt 
P-2S69.  The  aaterial  covered  has  been  of  vital 
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importance  in  the  development  of  the  HclansicalH 
operating  system,  yet  it  is  difficult  to  find  an 
adequate  treatment  outside  of  very  long  and 
usually  dry  system  manuals.  George  Healy  was  one 
of  the  few  working  experts  in  the  field  who  took 
the  time  to  write  down  soae  of  the  basic 
principles  of  operating  systems  and  also  cf 
assembly  systems." 

Hr.  Rosen’s  observations  imply  that  very  little 
attention  has  been  expended  in  the  atteapt  to  generalize  the 
functions  of  operating  systems.  Pile  systeis  have  also  been 
severely  neglected. 

In  the  early  years  of  computing  (rough.'y  1952-  1962), 
programmers  slowly  moved  away  froa  the  practice  of 
approachlnq  a  bare  aachine  with  card  decks  and  sharpened 
pencils,  fighting  with  the  console  for  aore  or  less  extended 
periods  of  time,  and  leaving  triumphantly  with  final  results 
or  in  defeat  with  a  ream  of  aachine  duap<Bosin  69>. 
Operating  systems  have  evolved,  not  so  much  as  a  blessing, 
but  as  a  practical  necessity.  As  computers  became  faster  acd 
•ore  complex,  it  vac  no  longer  possible  for  an  individual 
programmer  to  be  an  expert  in  every  phase  of  the  prcgcaixing 
and  tachine  usage;  he  now  must  rely  on  the  operating  staff 
and  system  progra iters  to  provide  the  necessities  of  life. 

These  operating  systems  were  often  ill-designed  and 
usually  specialized  around  a  single  goal.  One  of  the  first 
trul  >  successful  operating  systems  was  PHS  (FCBT3AR  Monitor 
System)  for  t.he  IBM  709/7090/7094  family.  Its  name  implies 
its  specialization.  As  a  result  a  large  number  of  operating 
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systems  appeared,  each  with  its  own  operating  procedure  and 
specialization.  These  sycteas  were  typically  very  clccely 
tied  to  a  programing  language  (e.g.  P0RTR1K ,  COBOL, 
Assembler) . 

Input  Output  Control  systeas  (XOCS)  emerged  as  a  pact 
of  the  Operating  Syatea  based  on  the  simple  observation  that 
all  prograas  perfora  soae  amount  of  input  and/or  output. 
Therefore,  rather  than  requiring  each  prograaaer  to  write  a 
new  set  of  input/output  routines  for  each  prograa,  a  ccaaon 
and  sufficiently  flexible  collection  of  routines  were 
supplied  with  the  Operating  Systea.  This  situation  became 
especially  critical  as  computer  I/O  capabilities  were 
extended  to  include  high-speed,  buffered,  asynchronous 
channels  which  required  cos  flex  prograa  logic  to  efficiently 
perfora  i nput- output . 

From  the  crude  beginnings  of  ICCS,  file  systems 
followed  a  logical,  though  often  slow,  evolution.  Once  all 
physical  input/output  functions  were  localized  in  the  IOCS, 
aany  generalizations  becaae  possible.  Usually,  there  is  no 
important  difference  aaong  the  aany  tape  drives  available  at 
an  installation,  sc  that  any  arbitrary  tape  unit  aay  be  used 
for  input  or  output  to  a  prograa.  Furthermore,  later  runs  at 
the  saae  or  different  installations  need  not  use  the  same 
unit  as  long  as  unique  correspondences  can  be  Maintained,  it 
first  it  was  considered  that  the  best  practice  in  handling 
the  choice  of  input-output  units  by  the  object  prograi  was 
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to  include  unit  assignments  as  an  assembly  parameter  cr  to 
read  in  unit  assignments  as  data  and  initialize  the  program 
appropriately.  this  practice  soried  well  when  it  was 
followed,  which  was  seldom.  Kith  the  advent  of  the 
near-universal  use  of  IOCS,  a  more  foolproof  and  flexible 
manner  of  operating  was  to  establish  the  correspondences  as 
part  of  the  IOCS*  The  object  prograis  dealt  strictly  in 
symbolic  unit  assignments. 

Since  the  object  programs  no  longer  interacted  directly 
with  the  I/O  units  not  were  even  aware  of  unit  assignments, 
additional  degrees  of  freedom  became  available  to  the 
operating  system,  providing  a  more  efficient  and  convenient 
environment.  For  example,  the  system  could  determine  unit 
assignments  automatically  and  dynamically,  based  upon 
complex  criteria  such  as  availability  and  performance  (e.  g. 
I/O  interference,  buffering,  etc.).  The  actual  technique  of 
I/O  (unbuffered,  single-buffered,  double-buffered,  etc.) 
could  be  removed  from  programmer  concern. 

The  proliferation  of  I/O  device  types,  such  as 
lov-speed,  aed iu m-speed,  high-speed  and  hyper-tapes,  as  veil 
as  drums  and  disks  of  all  shapes  and  sizes,  resulted  in  the 
expansion  of  IOCS  to  include  capabilities  that  are  now 
called  data  management  or  file  system  facilities.  The  basic 
notion  exploited  is  that  just  as  the  programmer  had  little 
concern  as  to  vhat  tapes  were  to  be  used,  he  really  does  not 
care  vhat  device  is  used  nor  vhat  method  of  I/O  is  employed 
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within  broad  logical  constraints.  For  example,  if  a 
programmer  wishes  logically  to  treat  his  I/O  data  as  80 
column  cards,  the  file  systes  could  physically  utilize 
unit-record  equipment,  tapes,  disks,  drums,  data  cells,  or  a 
host  of  other  devices  in  various  Banners  logically  to 
siaulate  the  effect  of  input-output  using  80  coluan  cards. 

This  trend  becane  irreversible  with  the  advent  of 
multi-tasking  operating  systeas,  since  the  availability  of 
devices  was  continuously  and  dynamically  changing.  In  such 
an  cnvironaent,  it  becoaes  impractical  and  probably 
iap'i visible  to  designate  specific  I/O  units  statically  and 
arbitrarily  in  the  program. 

The  importance  of  these  data  aanagement  and  file 
syst  .  cannot  be  overly  emphasized.  Just  as  the  assumption 
that  programs  perform  input-output  was  a  basic  fact,  it 
appears  that  the  number  and  flexibility  of  I/O  facilities 
demanded  by  programs  are  continuously  increasing. 

A  major  factor  in  the  rapid  growth  of  file  systems  is 
the  introduction  of  low  cost,  high  capacity,  high-speed, 
direct  access  devices  such  as  disks,  druas,  and  data  cells. 
A  description  of  direct  access  devices  would  eaphasize  the 
fact  that  they  have  two  degrees  of  freedom  rather  than  only 
one  as  with  tape-like  devices.  Since  these  devices  can  be 
used  for  both  sequential  and  direct  access  applications,  the 
total  amount  of  usage  increases.  Of  course,  the  extra 
degrees  of  freedom  necessitate  more  complex  I/O  routines  and 
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further  tighten  the  reliance  on  file  systeas  tc  perform 

these  functions. 

Direct  access  devices  ere  usually  as  flexible  as  or 
sore  flexible  than  tape  devices,  card-image  or  ptinter-iaage 
fixed  record  data  types  can  be  bandied  as  veil  as 
variable-length  or  structured  data  foras.  Although  these 
capabilities  could  be  perforaed  by  the  object  pregtaa,  the 
vast  aajority  of  these  functions  have  been  subsuaed  as 
by-products  of  the  file  systea. 

The  second  aa jor  factor  contributing  to  the  rising 

iaportance  of  file  systeas,  as  ir.  early  operating  systeas, 
vas  necessity.  This  tine  it  vas  due  to  the  "information 
explosion”.  As  the  nuaber  of  users,  uses,  and  sophistication 
of  use  increased,  the  aaount  of  information  in  the  feras  of 
programs  and  data  rose  correspondingly.  It  Mas  no  longer 
convenient  nor  usually  physically  possible  to  haul  the 
required  boxes  of  programs  and  data  to  and  froa  the  machine. 
This  inforaation  vas  converted  and  aaint&ined  in  a  more 

compact  but  directly  machine  processible  fora,  such  as 
aagnetic  tape  or  disk  pack.  Not  only  vere  the  individual 
prograas  and  data  collections  large,  but  the  total  nuaber  of 
distinct  and  unique  filets  (i.e.  prograas  and  data 

collections)  vas  very  large.  It  is  not  uncommon  for  a  single 
programmer  to  have  to  use  froa  10  to  100  separate  prograas 
and  a  roughly  equivalent  nuaber  of  data  collections.  Ihis 
situation  becaae  especially  acute  vith  the  increased  use  of 
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online  system.  1  user  at  a  cesote  teletype  terminal  could 

not  be  expected  to  re-type  and  enter  all  his  program  and 

data  from  the  terminal.  They  »ust  be  permanently  aaistained 

and  stored  at  the  central  computer  facility,  although 

accessible  and  alterable  under  remote  terminal  central. 

Quite  obviously,  it  would  be  uneconoaic  and  uneanagcable  to 

store  each  unique  file  on  a  separate  tape  or  dish  pack. 

Robert  Rosin  highlights  these  developments  in  his  recent 

survey  of  supervisor  and  aonitor  systeas<Rosin  6<5>: 

"i  file  systea  is  especially  necessary  in  any 
system  which  purports  to  provide  realistic 
t ime-sharing .  However,  the  advantages  of  this 
facility  cannot  be  overlooked  in  a  acre 
conventional  environment" . 


Thus,  people 

were  faced  with 

the 

problem  of  using 

the 

I/O  devices 

to 

store  thousands 

of 

permanent  files 

in 

addition  to 

the 

traditional  use 

for 

input,  output 

and 

"scratch"  storage.  Direct  access  devices  provide  the 
capability  of  storing  hundreds  or  thousands  of  unigue  files 
and  accessing  them  in  any  order  conveniently.  This  type  cf 
direct  access  device  usage  results  in  many  side  effects.  The 
first  problem,  of  course,  involves  a  complex  storage 
organization  facility  to  locate  "empty"  space  on  the  device 
and  a  directory- like  mechanism  to  keep  track  of  the 
individual  files.  Many  other  facilities  are  usually 
required,  such  as  a  security  systen  to  prevent  unauthorized 
access  to  restricted  files,  and  procedures  to  recover  from 
hardware  or  software  failures.  Of  course,  each  installation 
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or  group  develops  additional  elegant  file  system 
capabilities  to  aeet  special  requirements  or  to  provide 
extensive  flexibility. 

For  the  sane  reasons  that  programmers  utilize  and  rely 
on  the  file  system,  the  operating  systea  uses  the  facilities 
of  the  file  systea.  Foe  example,  user  identification  {e.g. 
passwords,  account  numbers,  etc.),  accounting  and  charge 
infornation  as  well  as  systea  self-seasuresent  data  aust  be 
Maintained  dynamically  using  the  facilities  of  the  file 
systea.  The  previously  aenticned  directories  of  "empty" 
space  on  direct  access  devices  aud  the  symbolic  file 
directory  and  access  control  information  are  usually  handled 
as  system  files.  The  operating  system  uses  the  file  system 
capabilities  to  store  the  various  processing  programs  (e.g. 
FORTRAN,  COBOL,  Assemblers,  etc.)  as  well  as  many 
infrequently  used  supervisor  routines.  Furthermore,  advanced 
operating  systeas  perform  "spooling'',  roll-in/roll-out,  and 
paging  in  conjunction  with  the  file  system.  It  is  not  hard 
to  realize  that  the  file  system  is  usually  the  most 
iaportant  component  of  an  operating  system  in  terms  of  the 
manpower  required  to  develop  and  implement,  and  the  ascust 
of  instructions  and  space  used  by  the  file  systei. 

Whereas  the  early  operating  systeas  along  with  their 
rudiaentary  file  systeos  revolved  around  the  need  tc  support 
miscellaneous  I/O  functions  for  prograaaing  languages, 
aodern  file  systems  are  at  the  very  center  of  the  operating 
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systea.  The  supervisor,  prograaaing  systeas,  and  object 
prcqraas  are  totally  dependent  on  the  file  systea. 
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Scope  §pd  faE£2S« 

The  developaent  of  file  systeas  has  suffered  free  easy 
of  the  save  preblees  as  that  of  programing  languages, 
probably  the  single  test  important  protlea  vas  the  excessive 
concern  with  efficiency.  Of  course  efficiency  is  important# 
but  in  aost  current-day  prograaaing  situations  other 
factors,  such  as  productivity  and  flexibility,  are  finally 
receiving  their  long-deserved  attention.  The  question  of 
efficiency  can  be  put  into  proper  perspective  froa  recent 
studies  of  real  prograaaing  groups,  where  it  has  been  found 
that  the  "best"  prograaner  was  up  to  15  tines  sore 
"efficient"  than  the  least  proficient  prograaner.  It  is  not 
the  function  of  this  paper  to  get  deeply  involved  in 
prograaaing  language  controversies,  but  to  illustrate  the 
trends  and  changing  attitudes.  For  exanple,  if  the  original 
designers  of  POBlBiN  had  not  felt  that  its  acceptance 
depended  on  the  utaost  attention  to  efficiency  and, 
therefore,  had  not  defined  the  language  in  terns  of  the 
hardware  capabilities  of  a  specific  Machine,  IBM  7Ctt,  it  is 
possiMf  that  the  evolution  of  languages  such  as  F0HTB1H-IV, 
COBOL,  ALGOL ,  and  PL/I  and  generalised  coapiler  techniques 
sight  nave  proceeded  in  a  acre  organised  fashion.  The  entire 
field  of  generalized  approaches  tc  progranning  languages  and 
compiler  techniques  has  only  recently  eaerged  as  a  najor 
factor  in  the  coaputing  profession. 
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file  ayateas  have  followed  a  similar  development.  In 
the  nave  of  "efficiency",  each  nev  file  system  van  specially 
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deaands  on  a  given  file  systea  increased,  new  features  and 
facilities  mere  added,  often  with  a  "crovbar".  each  cf  these 
piece-seal  file  syeteas  drove  ns  further  and  further  frea  an 
organised,  generalised  file  syetea  structure. 

(lost  literature  in  this  area  has  appeared  in  one  of  teo 
fores.  The  typical  systea  annuals  describe  the  "clever" 
techniques  used  to  iapleaent  a  specific  file  systea,  but 
provide  very  little  assistance  for  comparisons  with  ether 
current  systeas  or  in  the  design  of  nea  file  systeis.  The 
other  type  of  reference  deals  uith  discussions  of  desirable 
characteristics  for  future  file  systeas,  usually  eaphasizing 
user  facilities,  but  adds  little  insight  into  the  problems 
of  designing  and  ispleaenting  such  a  systea. 

To  a  certain  extent,  generalised  approaches  have  begun 
to  evolve  in  "tine-sharing"  systeas.  in  this  paper  such 
systems  will  be  called  conversational  resource-sharing, 
since  tine  is  only  one  of  many  resources  that  are  shared  and 
it  is  the  conversational  or  interactive  nature  cf  these 
systems  that  is  aost  easily  distinguished  froa 
batch- or iented  operating  systeas. 

These  generalized  file  systems  for  conversational 
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resoure*-s ‘■nr ing  operating  systems  dmvmlopmd  both  by  design 
and  necessity.  In  order  to  provide  all  the  features  required 
by  user  programs  end  the  supervisor,  a  flexible  design  sea 
essential.  Furthermore,  owing  to  the  complexity  of  the 
enwi ronsent  and  its  dynamically  changing  aspects,  it  would 
be  impossible  to  devise  an  "optimally  efficient"  strategy. 
The  implemented  mere  thus  forced  to  abandon  any  attempt  to 
make  the  system  mere  efficient  and  vere  free  to  develop  a 
flexible  system  with  a  cleat  conscience. 

The  goals  of  flexibility  and  efficiency  need  not  be 
contradictory.  In  any  multi- tasking  system,  which  includes 
most  modern,  non-conversat. ional ,  batch-oriented  operating 
systems  as  well  as  conversational  systems,  I/C  operations 
can  be  performed  asynchronously  by  channels,  and  the  central 
processor  time  can  be  utilized  by  executing  other  tasks 
while  t/C  is  in  progress.  In  this  environment  file  system 
efficiency  ceases  to  be  of  paramount  concern.  Furthermore, 
individual  user  attempts  to  optimize  performance  could 
result  in  unnecessary  inefficiencies  due  to  conflicts  with 
other  tasks,  such  as  excessive  I/O  interference  from 
overloading  the  channels.  The  file  system,  aware  of  the 
total  requirements,  could  provide  a  strategy  that  results  in 
a  aore  harmonious  arrangement,  increasing  system  throughput 
far  lot*’  than  individual  user  optimization  could. 

Fven  in  single-task  cr  application- criented  operating 
■;  y  st  *  a  ,  there  is  definite  value  to  an  organized. 
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i)«ii«{dln4  file  system.  for  most  large,  complex  user  9 

programs  as  veil  as  compilers  and  assemblers,  the  program  9 

action  including  precise  file  system  requ  ireaccts  catinct  be  9 

statically  determined  since  it  is  a  dynamic  tuncticn  cf  the  9 

input  data  supplied.  Therefore,  a  dynamically  flexible  file  9 

,  system  could  often  outperform  a  tpeeialized,  but  inflexible,  9 

I  file  system. 

;■  Xt  is  the  purpose  of  this  paper  to  present  a  general 

|  file  system  design.  Xt  is  extremely  important  to  start  with 

:  a  flexible  but  precise  model  although  this  design  will 

probably  need  to  be  modified  and  made  sore  detailed  fer  any 
specific  implementation.  This  issue  was  highlighted  by 

Robert  Rappaport  in  his  thesis  ‘'Implementing  tlulti-Piccoss 

I 

Primitives  in  a  Multiplexed  Computer  Systea"<Rapp  €e>  which 
describes  the  development  of  the  Traffic  Controller  for  the 
HIT  Project  HAC  Hultics  Systei: 

"lifter  having  found  acceptable  solutions  for 
the  problems  at  hand,  one  asks  oneself  why  it  tcck 
so  long  to  arrive  at  these  solutions  and  was  there 
any  way  to  have  done  it  more  quickly?  One  aight 
further  ask  if  the  arrived-at  solutions  are  in  any 
I  sense  optimum? 

After  being  involved  in  designing  a  large 
system  involving  the  work  of  many  people,  one  gets 
the  feeling  that  such  problems  as  were  encountered 
here  are  bound  to  crop  up.  The  development  cf  any 
large  system  can  only  remain  manageable  if 
distinct  parts  of  the  system  remain  modular  and 
independent, 

Without  a  theory  of  computing  systems  tc  fall 
back  on,  designing  such  complex  systems  becomes  an 
art,  rather  than  a  science,  in  which  it  is 
impossible  tc  prove  the  degree  to  which  working 
solutions  to  problems  are  in  any  sense  optima 
solutions.  In  much  the  same  way  as  authors  write 
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books,  large  conputer  systees  go  through  ■•vocal 
drafts  before  they  begin  to  take  shape.  To  the 
absence  of  a  theory  one  can  only  cope  with  the 
coiplexity  of  the  situation  by  proceeding  in  an 
orderly  fashion  to  first  produce  ea  initial 
sorting  aodel  of  the  desired  systea.  This  part  of 
the  work  represents  the  aajor  effort  of  the  decigs 
and  isplesentation  project.  Once  having  arrived  at 
this  benchmark,  easy  of  the  problems  say  this  be 
seen  in  a  clearer  light  and  revisions  to  the 
working  eodel  are  ispleaented  such  sore  guickly 
than  were  the  originel  eodules.  is  to  the 
developeent  of  a  theory,  one  gets  the  impression 
that  it  will  be  a  long  tine  in  coning." 

Therefore,  while  we  await  THE  general  theory  of 

coeputer  science,  the  file  syatea  eodel  presented  in  this 

paper  will  hopefully  serve  the  need  for  an  "initial  working 

eodel"  fron  which  "probless  eay  be  seen  in  a  clearer  light". 
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CHAPtEB  THO 

Motivation  Behind  Pile  Syaten  Design 

There  ere  two  basic  goals  to  be  satisfied  by  the  file 
systos  design.  It  is  necessary  to  (1)  establish  a  unifora 
representation  of  a  file's  structure  and  (2)  determine  the 
hierarchy  of  logical  transforaations  that  occur  in  a  file 
systea.  ».  B.  Henry's  recent  paper  on  hierarchical  data 
aanageaent  systeas<Benry  69>  discusses  siailar  noticns  cf 
separating  logical  and  physical  file  control,  but  differs 
significantly  froa  the  approaches  presented  in  this  report 
in  aany  fundaaental  says.  It  should  be  a  useful  reference  to 
a  reader  interested  in  other  current  research  in  this  area. 

u&1£££!  JtefiissanUiiaa  stt  fils  ittasi axe 

A  typical  coaputer  systea  is  portrayed  by  Figure  2.1. 
Such  a  configuration  usually  has  a  varied  assortment  cf 
secondary  storage  devices  in  addition  to  the  primary 
storage.  Prograas  and  data  aust  be  in  priaary  stcrage  in 
order  to  be  executed  or  operated  upon,  respectively. 

It  is  generally  true  that  if  priaary  storage  sire  was 
liaitless  and  very  inexpensive,  there  mould  be  no  need  for 
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secondary  storage  (possible  exceptions  say  be  backup 
requirements  and  tcansfoc  of  data).  In  the  fraiewotk  of  this 
report,  the  file  systea  will  be  defined  as  the  software 
aechanisa  that  extends  the  capacity  of  prieary  stcragj  by 
handling  and  coordinating  the  transfer  of  inforaatien  tc  and 
froe  the  secondary  storage  devices.  This  definition  is 
soaevhat  aore  restrictive  than  other  coaaon  interpretations 
which  include  as  fart  of  the  file  systea  the  physical 
devices  or  the  ptograas  that  operate  upon  the  data.  In  this 
interpretation  the  file  systea  aerel  stores  and  transfers 
infocaation  but  does  not  operate  upon  it. 
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Figure  2.  1 

Physical  Coiputez  Cos  figuration 
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XI.  HOTIUTION  BEHIND  PIIE  SYSTIB  DESIGN 


Early  file  systeas  were  usually  designed  to  operate 
with  specific  application  programs.  Since  there  are 
potentially  a  very  large  number  of  different  secondary 
storage  devices,  aany  of  which  can  be  used  in  aore  than  one 
way  (e.g.  sequential  or  tandoa  access,  blocked  or  unblocked, 
etc.)  each  file  systea  liaited  itself  specifically  to  those 
devices  and  organirations  that  were  appropriate  fer  its 
intetded  application,  figure  2.2  depicts  the  relationships 
between  the  applications,  the  devices,  and  the  file  systems. 

This  type  of  developaent  produced  chaotic  situations. 
It  is  sonevhat  analogous  to  assembly  language  programing 
without  any  established  standard  calling  sequences  or 
conaunication  conventions,  which  makes  it  difficult,  if  not 
impossible,  to  use  arbitrary  programs  as  subroutines.  In 
particular,  it  was  quite  common  to  find  that  data  files 
produced  by  the  payroll  programs,  using  their  private  file 
system,  could  not  be  accessed  by  the  file  systea  used  by  the 
personnel  progtars,  and  vice  versa.  is  a  result,  there  was 
such  duplication  of  effort  and  confusion  in  the  development 
and  use  of  these  early  file  systems. 
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Figure  2.2 
Early  Pile  systeas 

(Analogous  to  Assembly  Language  Programing) 
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II.  ROTIMTXOI  BERT  ID  THE  STSTtR  EESIO* 


Rote  recently,  the  computer  aanu lecturers  and  operating 
systca  designers  realized  that  it  is  possible  to  select  a 
aaall  set  of  coason  logical  file  organization*  (or  access 
sethods)  that  can  satisfy  the  needs  of  sost  application 
prograss.  Furthermore,  these  access  aethoda  could  he 
designed  in  a  flexible  aanner  to  operate  on  a  variety  of 
different  devices  and  device  organizations.  This  provided 
the  user  sith  a  logically  device-independent  interface  with 
the  file  systen.  Figure  2.3  illustrates  this  structure. 

This  approach  can  be  ccspared  sith  the  emergence  cf 
Problea  Oriented  languages,  such  as  COBOL  for  business 
applications  and  FORTRAN  for  scientific  applications.  The 
access  sethods  file  systeas  suffered  the  same  shortcosings 
as  the  prograseing  languages:  (1)  despite  claiss,  they  were 
not  really  device  independent,  (2)  occasionally  it  was 
necessary  to  resort  to  asseably  language  to  overcose  cr 
bypass  a  restriction,  and  (3)  it  was  not  possible  to 
inter-six  access  sethods  (analogy  vould  be  to  intersix 
FORTRAN  and  COBOL  subroutines) . 
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APPLICATIONS 


DEVICES  AND  PHYSICAL  DATA  ORGANIZATION 


Figure  2.3 

Access  Methods  File  Systeas 


(Analogous  to  Early  Programing  Languages, 
Such  as  FOBTRA  N  and  COBOL) 
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IX.  ftOTIVlTZOX  BBUIBD  FILE  SISTER  CtSZCK 


In  order  to  overcoae  the  weakness  in  the  acceas  aethoda 
approach,  it  is  necessary  to  design  a  single  unifcra  file 
representation  that  can  <1)  be  need  for  every  application 
and  {2}  be  device  independent.  this  idealistic  goal  is 
analogous  to  the  search  for  "The"  universal  prograeeing 
language,  for  which  PL/1  is  probably  the  aost  aabitious 
atteept  to  date. 

It  is  reasonable  to  expect  that  such  a  unifcra 
representation  will  be  ao  atosic  or  primitive  in  fere  that 
it  Mill  be  desirable  tc  construct  sore  powerful  specialized 
access  aethods  fer  the  convenience  of  the  typical  user. 
Since  the  access  aethods  are  built  upon  the  unifora 
representation,  it  is  auch  easier  to  aodify  or  iapleaent  new 
access  aethods  or,  if  necessary,  operate  at  the  atonic  level 
to  bypass  the  restrictions  of  the  access  aethods.  This 
approach  pushes  the  logical/physical  separation  of  file 
system  structure  auch  further  as  indicated  in  Figure  2*4, 
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APPLICATIONS 


DEVICES  AND  PHYSICAL  DATA  CIG ANIMATION 


Figure  2.U 

Unitors  File  Representation 


(Analogous  to  Universal  Programing  Language, 

PL/I  ?) 
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The  rationale  behind  the  selection  of  a  particular 
unifora  representation  is  not  trivial.  For  exaapla,  there 
are  three  broad  classes  of  coaaon  unifera  representation*: 

1.  Stress  -  every  file  is  treated  as  a  continuous 
sequential  stroaa  of  inforaation.  It  is  possible 
to  access  only  the  current  position  in  the  stress 
or  reposition  to  the  beginning  of  the  street,  this 
representation  can  be  ispleaented  conveniently  on 
alsost  all  secondary  storage  devices,  although  it 
does  not  provide  the  user  with  very  powerful  or 
efficient  features  for  nany  applications. 

2.  Direct-iccess  -  every  file  is  treatod  as  an 
ordered  collection  of  itess.  Each  ites  is  directly 
accessable  by  leans  of  a  unique  identifier 
corresponding  to  its  position  in  the  ordering. 
This  representation,  which  corresponds  to  priaary 
storage  organization,  is  aore  powerful  than 
Streaa,  but  is  very  difficult  to  isplesent  on 
intrinsically  serial  devices,  such  as  sagnetic 
tape . 

3.  Associative  -  every  file  is  treated  as  an 
unordered  collection  of  itess,  each  itea  is 
directly  accessible  by  seaas  of  an  identifier  that 
has  been  "associated*  with  the  itea.  This  is  a 
very  flexible  representation.  Unfortunately, 
except  for  a  saall  class  of  sophisticated 
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secondary  storage  devices  the  iiplesentaticn  is 
vary  ccsplex  and  inefficient. 

Irtegardless  cf  the  specific  unifora  representation 
chosen,  the  iaportant  concept  is  that  all  files  can  be 
viewed  an  being  identical  in  structure  independent  cf  the 
particular  physical  device  on  which  the  file  ia  recorded. 
This  generalization  is  depicted  in  Figure  2.5,  which  should 
be  coapared  with  Figure  2.1. 
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Logical  Confutes  Configuration 
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Although  «  precise  description  of  a  file  systea  will 
not  ie  presented  until  later  sections,  there  are  several 
general  characteristics  of  most  file  systems.  In 
particular,  a  user  specifies  his  reguest,  such  as  read  or 
write,  by  designating  a  file  and  an  element  within  the  file. 
Host  advanced  file  systess  allow  considerable  flexibility  in 
the  Mechanise  used  to  specify  a  file,  it  is  typically 
described  by  leans  of  a  symbolic  file  name.  Furthermore,  the 
elesent  within  the  file  is  specified  in  teres  of  the  logic*’ 
representation  of  elesents  in  the  particular  file  systes 
which  say  or  say  not  correspond  to  a  precise  physiol 
specification  cf  hew  and  where  the  elesent  is  stored,  fci 
example,  a  typical  request  eight  be  of  the  fore: 

"Head  item  23  from  file  A I PH A  into  location  1564." 

Realising  that  information  must  usually  be  stored  on 
devices  in  sosewhat  obscure  ways,  there  must  be  roe 

sequence  of  transformations  required  to  convert  the  user's 
request  into  its  final  form  that  physically  operates  ct  the 
secondary  storage  device.  Quite  often  the  transf  orvat  i  m  i:, 
viewed  as  a  single  step  but  that  is  a  gtoss 

oversimplification  that  hides  the  fundamental  sechan isss  hi 
use.  In  Figure  2.6  the  conversion  process  is  i  11  ufti  >»  m 
terms  of  a  discrete  sequence  of  logical  transf  or  mat  xon.-.. 

Since  the  specifics  of  these  transforaat  ions;  *  i  «  ret  he 
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Logical  Transformations  in 


File  System 
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obvious  until  the  »ote  detailed  sections  later,  a  si* tie 
analogy  is  presented  in  Figure  2.6  that  loosely  parallels 
the  file  system  transformations.  The  analogy  is  only 
intended  to  provide  soae  insight  into  the  rationale  behind 
each  stage  of  the  transf oraa tion. 

The  process  starts  froa  the  usec*s  request  to  "read 
itea  23  froa  the  file  ALPHA  into  location  1564”.  The  first 
step  is  to  convert  the  syabolic  file  naae  into  a  unique 
nuueric  file  identifier.  In  the  analogy,  this  corresponds  to 
looking  up  John  Dee's  identifier  which  is  a  social  security 
in  this  illustration.  The  purpose  for  using  an  identifier  is 
basically  the  saae  in  both  cases.  It  is  usually  aore 
convenient  to  store  information,  manually  or  automatically, 
by  means  of  a  unique  nuaeric  "key"  rather  than  a  syabolic 
naae  which  may,  under  certain  circu astances,  not  even  be 
unique  (i.e.  there  may  be  aore  than  one  John  Doe  in  uhich 
case  other  factors  must  be  considered  in  order  to  uniguely 
identify  the  person  under  consideration). 

The  file  identifier  can  then  be  used  to  conveniently 
access  all  the  inforaaticn  known  about  a  file,  this 
information  collectively  is  known  as  the  file's  descriptor. 
In  the  analogy,  this  would  correspond  to  requesting  all 
inforaation  in  the  social  security  records  of  030-34-1234. 

Now  that  everything  is  known  about  the  file,  it  is 
npcessary  to  consider  the  specific  operation  tc  be 
performed.  Using  the  file  descriptor. 


a  sequence  cr  lcgical 
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I/O  coaaands  can  be  produced.  These  are  called  logical  I/O 
coaaands  because  they  do  sot  consider  the  physical 
characteristics  of  the  secondary  storage  device  to  be  used. 
This  is  analogous  to  putting  an  address  on  a  letter  which  is 
usually  done  without  considering  the  physical  destination 
nor  the  route  to  be  taken. 

In  order  to  coaplete  the  transforaation,  the  logical 
I/O  coaaands  Bust  be  converted  into  the  appropriate  sequence 
ot  physical  I/O  comands.  This  conversion  aay  be  trivial  or 
coaplej  depending  upon  the  peculiarities  of  the  device  and 
I/O  interfaces  to  the  devices.  In  the  analogy  this  process 
is  performed  at  the  post  office  where  the  address  is  used  to 
deteraine  the  physical  routing  needed  to  get  the  letter  to 
its  destination. 

The  final  step  in  the  process  is  the  physical  transfer 
of  information .  This  is  usually  perforaed  by  aeans  of 
sof twa re /hardware  interactions  to  activate  the  appropriate 
device  and  confira  the  successful  coapletion  of  the  request. 
Of  course,  in  the  analogy  this  transfer  is  accomplished  by 
the  postman  ("neither  rain  nor  snow  nor  dark  cf  night...") 
assisted  by  trucks,  planes,  trains  and  other  autoaaticn* 
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C  HA  PIER  THREE 

Pile  Systes  Design  Model 

Basis  Concepts  ns^d  Ijj  £iie  s vstea  pesjgn 

Two  concepts  ace  basic  to  the  general  file  systea  aodel 
to  be  introduced.  These  concepts  have  been  described  by  the 
teras  "hierarchical  Modularity'*  and  "virtual  aeaory".  They 
will  be  discussed  briefly  below. 

Hierarchical  Nodularity 

The  tera  "aodularity"  leans  aany  different  things  to 
different  people.  In  the  context  of  this  paper  we  will  be 
concerned  with  an  organization  siiilac  to  that  proposed  by 
Di jkstra<Di jks  67><Dijks  68>  and  RandelKRand  68>.  The 
iaportant  aspect  of  this  organization  is  that  all  activities 
are  divided  into  sequential  processes.  A  hierarchical 
structure  of  these  sequential  processes  results  in  a  level 
or  ring  organization  wherein  each  level  only  coaounicates 
with  its  iaaediately  superior  and  inferior  levels. 

The  notions  of  "levels  of  abstraction"  or  "hierarchical 
aodularity"  can  best  be  presented  briefly  by  an  exasple. 
Consider  an  aeronautical  engineer  using  a  aatrix  inversion 


12 


III.  FILE  S YSTEfl  DESIGN  HCDZL 


pacKagf  to  solve  space  flight  problems.  At  his  level  cf 
abstraction,  the  computer  is  viewed  as  a  matrix  inverter 
that  accepts  the  satrix  and  control  information  as  input  and 
provides  the  inverted  satrix  as  output.  The  application 
programmer  who  wrote  the  aatrix  inversion  package  need  not 
have  had  any  knowledge  off  its  intended  usage  (superior 
levels  of  abstraction).  He  sight  view  the  computer  as  a 
"FORTRAN  machine",  for  example,  at  his  level  of  abstraction. 
He  need  not  have  any  specific  knowledge  of  the  internal 
operation  of  the  FORTRAN  system  (inferior  level  cf 
abstraction),  but  only  of  the  way  in  which  he  can  interact 
with  it.  Finally,  the  FORTRAN  compiler  implementor  operates 
at  a  different  (lower)  level  of  abstraction.  In  the  above 
example  the  interaction  between  the  3  levels  of  abstraction 
is  static  since  after  the  matrix  inversion  program  is 
completed,  the  engineer  need  not  interact,  even  indirectly, 
with  the  applications  programmer  or  compiler  implementer.  In 
the  form  of  hierarchical  modularity  used  in  the  file  system 
design  scdel,  the  sulti-level  interaction  is  continual  and 
basic  to  the  file  system  operation. 

There  are  several  advantages  to  such  an  modular 
orianivation.  Possibly  the  most  important  is  the  logical 
completeness  of  each  level.  It  is  easier  for  the  system 
designers  and  impleaenters  to  understand  the  functions  and 
interactions  of  each  level  and  thus  the  entire  system.  This 
is  cften  a  very  difficult  problem  in  very  complex  file 


BASIC  CONCEPTS 


1 

V 

4 - - — —  ♦ 

|  LEVEL  KO  | 
4 - ♦ 

I 

V 

> - 4 

|  LEVEL  X  | 
♦ - ♦ 

I 

V 

4  4 

|  LEVEL  X-1  | 

4 - - 

\ 

V 


Figure  3.  1 
Hierarchical  Levels 


Itt 


III.  PUB  STSTBH  OBSIGP  BODEl 


systeas  with  tens  or  hundreds  of  thousands  of  instructions 
anl  hundreds  of  inter-dependent  routines. 

Another  by-product  of  this  structure  is  "debugging" 
assistance.  For  esaaple,  when  an  error  oceurs  it  can  usually 
be  local  i*ed  *t  a  level  and  identified  easily.  The  cosplet* 
verification  (reliability  checkout)  of  a  file  systes  is 
usually  an  iapossible  task  since  it  would  require  tests 
using  all  possible  data  input  and  systes  requests  occuring 
in  each  potential  "systes  state".  In  order  to  construct  a 
finite  set  of  relevant  tests,  it  is  necessary  to  consider 
the  internal  structure  of  the  sechaniss  to  be  tested. 
Therefore,  an  isportant  goal  is  to  design  the  infernal 
structure  so  that  at  each  level,  the  nunber  of  test  cases  is 
sufficiently  saall  that  they  can  all  be  tried  without 
overlooking  an  ispcrtant  situation.  In  theory,  level  0  would 
be  checked-out  and  verified,  then  level  1,  level  2#  etc., 
each  level  being  aore  powerful,  but  because  of  the 
abstractions  introduced,,  the  nuaber  of  "special  cases" 
regains  within  bounds. 

Virtual  Memory 

There  are  four  very  isportant  a  no'  difficult  file  systea 
objectives:  (1)  a  flexible  and  versatile  foraat,  (2)  as  auch 
of  the  aechanisa  as  possible  should  be  invisible,  (3)  a 
loqree  of  machine  and  device  independence,  and  (4)  dynaaic 
an?  automatic  allocation  of  secondary  storage.  There  have 
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been  several  techniques  developed  to  satisfy  these 
objectives  in  an  organized  tanner;  the  concept  exploited  la 
this  generalised  file  systes  has  bees  called 
•‘segmentation**  Denn  6S>  or  **naaed  virtual  aetory H<Dal«y  68>. 
Under  this  systea  each  file  is  treated  as  an  ordered 
sequence  of  addressable  elements,  where  each  detent  is 
aoraally  the  aaae  size  unit  as  the  tain  storage,  a  byte  or 
word.  Therefore,  each  individual  file  has  the  fora  of  a 
"virtual**  core  aetory,  froa  thence  the  naae  of  the  technique 
caae.  the  size  of  each  file  is  allowed  to  be  arbitrary  and 
can  dynaaically  grow  and  shrink,  there  is  no  explicit  data 
foraat  associated  with  the  file;  the  basic  operations  of 
the  file  systea  aove  a  specified  nuaber  of  eleaents  between 
designated  addresses  in  "real**  aetory  and  the  "virtual** 
aenory  of  the  file  systea. 

There  are  several  reasons  for  choosing  such  a  file 
concept.  In  scae  systems  the  similarity  between  files  and 
•ain  storage  is  used  to  establish  a  single  mechanist  that 
serves  as  both  a  file  systea  for  static  data  and  prograa 
storage  and  a  paging  systea<Lett  68><Daley  68><Denn  6BXSalt 
6fl>  for  dyaaaic  storage  sanageaent.  "Virtual  aetory" 
provides  a  very  flexible  and  versatile  foraat.  then 
specific  formatting  is  desired,  it  can  be  accomplished  by 
the  outermost  file  systea  level  or  by  the  user  pccgiaa.  Per 
exaaple,  if  a  file  is  to  be  treated  as  a  collection  of 
cacd-iaage  records,  it  is  aerely  necessary  to  establish  a 
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routine  to  access  60  characters  at  a  time  starting  at  byte 
locations  0,  80,  160,  ...  •  Almost  all  other  possible 

formats  can  be  realized  by  similar  procedures. 

Except  for  the  formatting  modules,  the  entire  file 
system  mechanism,  including  allocations,  buffering,  and 
physical  location,  is  completely  hidden  and  invisible  tc  the 
user.  This  relates  closely  to  the  objective  of  device 
independence,  in  sany  file  syatena  the  user  sust  specify 
which  device  should  be  used,  its  record  site  (if  it  is  a 
hardware  forsatable  device)  ,  blocking  and  buffering  factors, 
and  sometimes  even  the  physical  addresses.  Although  the 
parameters  and  algorithms  chosen  sight,  in  soae  sense,  be 
optiaal,  sany  changes  night  be  necessary  if  the  prog  ran  is 
required  to  run  with  a  different  configuration  or 
environment.  This  strategy  does  not  prevent  the  user  frea 
providing  additional  information,  such  as  how  often  the  file 
will  be  used  and  in  what  Banner.  The  important  factor  is 
that  this  infozsation  is  not  necessary  and  its  significance 
is  determined  by  the  file  system  rather  than  the  user. 

There  are  very  serious  questions  of  efficiency  raised 
by  this  file  system  strategy.  Host  of  these  fears  can  be 
eased  by  the  following  considerations.  First,  if  a  file  is 
to  be  used  very  seldoa  (as  in  program  development), 
efficiency  is  not  of  paramount  importance;  if,  on  the  ether 
hand,  it  is  for  long-term  use  (as  in  a  commercial  production 
program),  the  device-independence  and  flexibility  for  change 
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and  upkeep  will  be  very  important.  Second,  by  relieving  the 
programmer  of  the  eeeplevities  of  the  forests,  devices,  and 
allocations,  he  Is  able  to  utilize  his  energy  sore 
constructively  and  creatively  to  develop  clever  algoritbse 
relating  to  the  logical  structuring  of  hie  problem  rather 
than  clever  "tricks*  to  ovetcoie  the  shortcomings  or 
peculiarities  of  the  file  system.  Third,  in  view  of  the 
coepleiity  of  current  direct-access  devices,  it  is  quite 
possible  that  the  file  systes  will  be  better  able  to 
coordinate  the  files  than  the  average  user  attempting  to 
specify  critical  paraeeters. 
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aisiiisi  si  His  §msi  sisiu  n^dei 

Tb*  file  ayatew  design  aodel  to  be  presented  in  thin 
paper  can  be  viewed  as  a  hierarchy  of  seven  levels.  In  a 
specific  iapleaentation  certain  levels  eay  be  forther 
sub-divided  or  coibined  as  required.  A  recent  study  of 
several  aodern  file  syataas,  which  will  be  published  in  a 
separate  report,  attempts  to  analyze  the  systeis  in  the 
fraaework  of  this  basic  sodel.  In  general  all  of  the  systeas 
studied  fit  into  the  aodel,  although  certain  levels  in  the 
•odel  are  occasionally  reduced  to  trivial  fori  or  are 
incorporated  into  other  parts  of  the  operating  systea. 

The  seven  hierarchical  levels  are: 

1.  Input/Output  Control  Systea  <IOCS) 

2.  Device  Strategy  Modules  (DSN ) 

3.  Allocation  Strategy  Nodules  (ASH) 

4.  Pile  Organization  Strategy  Nodules  (POSH) 

5*  Basic  Pile  Systea  ( BPS) 

6.  Logical  Pile  Syatea  (LPSJ 

7.  Access  Methods  and  User  Interface 

The  hierarchical  organization  can  be  described  froa  the 
"top"  down  or  froa  the  "bottoa*  up.  The  file  systea  vculd 
ordinarily  be  iaplesented  by  starting  at  the  lowest  level, 
the  Input/Output  Control  Systea,  and  working  up.  It  appears 
aore  teaoingful,  however,  to  present  the  file  systea 
organization  starting  at  the  acst  abstract  level,  the  access 
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routines,  and  removing  the  abstractions  as  the  levels  are 
"peeled  avay“» 

In  the  following  presentation  the  teres  "file  sane", 
"file  identifier",  and  "file  descriptor"  sill  be  introduced. 
Detailed  explanations  cannot  be  provided  until  later 
sections,  the  following  analogy  §a,y  be  used  for  the  reader’s 
assistance.  A  person’s  nase  (file  naae)  ,  due  to  the  soaewhat 
haphazard  process  of  assignment,  is  not  necessarily  unique 
or  sanageable  for  computer  processing.  A  unique  identifier 
(file  identifier)  is  usually  assigned  to  each  person,  such 
as  a  Social  Security  number.  This  identifier  can  then  be 
used  to  locate  efficiently  the  intonation  {file  descriptor) 
known  about  that  person. 

Access  Methods  { AM) 

This  level  consists  of  the  set  of  »:«  utines  that 
superiapose  a  foraat  on  the  file.  In  general  there  will 
probably  be  routines  to  simulate  sequential  fixed-length 
record  iiiles,  sequential  variable- length  record  files,  and 
direct-access  f ixed- leng th  record  files,  fot  exanple.  Many 
■  ore  elaborate  and  specialized  fon<  routines,  also  called 
access  methods  or  data  management,  .  n  be  supplied  as  part 
of  the  file  systea.  obviously,  a  user  may  write  bis  own 
access  aethods  to  augaent  this  level. 
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Logical  File  System  (LP  S) 

Routines  above  this  level  of  abstraction  associate  a 
symbolic  nan©  with  a  file.  It  is  the  function  of  the  Lcgical 
File  Systea  to  use  the  symbolic  file  name  to  find  the 
corresponding  unique  "file  identifier".  Belov  this  level  the 
symbolic  file  naie  abstraction  is  eliminated. 

Basic  File  Systee  (BPS) 

The  Basic  File  Systee  nust  convert  the  file  identifier 
into  a  file  descriptor.  In  an  abstract  sense,  the  file 
descriptor  provides  all  information  needed  to  physically 
locate  the  file,  such  as  the  "length"  and  "location"  of  the 
file.  The  file  descriptor  is  also  used  to  verify  access 
rights  (read-only,  write-only,  etc.),  check  read/vrite 
interlocks,  and  set  up  system-vide  data  bases.  The  Easic 
File  System  performs  many  of  the  functions  ordinarily 
associated  with  "opening"  or  "closing"  a  file.  Finally, 
based  upon  the  file  descriptor,  the  appropriate  POSH  for  the 
file  is  selected. 

File  Organization  Strategy  Modules  (POSM) 

Direct- access  devices  physically  do  not  resemble  a 
virtual  memory.  A  file  must  be  split  into  sany  separate 
physical  records.  Each  record  has  a  unique  address 
associated  vith  it.  The  File  Organization  Strategy  Module 
maps  a  logical  virtual  memory  address  into  the  corresponding 
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physical  record  addrens  and  offset  within  the  record. 

To  read  or  write  a  portion  of  a  file,  it  is  necessary 
for  the  POSH  to  translate  the  lo9ically  contiguous  virtual 
aeaory  area  into  the  correct  collection  of  physical  records 
or  pcrtion  thereof,  if  necessary,  new  records  are  allocated 
by  tie  ASH.  The  list  of  records  to  be  physically  processed 
is  passed  on  to  the  appropriate  DSH. 

Althouqh  not  necessary,  the  POSH  is  often  designed  to 
allocate  "hidden"  file  buffers  in  order  to  sinisize 
redundant  or  unnecessary  I/O.  If  the  requested  pcrtion  of 
virtual  aeaory  is  contained  in  a  currently  buffeted  record, 
the  data  can  be  transferred  to  the  designated  user  sain 
storage  area  without  intervening  I/O.  Conversely  output  to 
the  file  aay  be  buffered.  If  a  sufficiently  large  nuaber  of 
buffer  areas  are  allocated  to  a  file,  it  is  possible  that 
all  read  and  write  requests  can  be  perforaed  by  Merely 
aoving  data  in  and  out  of  the  buffers.  When  a  file  is 
"closed",  the  buffers  are  eaptied  by  updating  the  physical 
records  on  the  secondary  storage  device  and  released  fer  use 
by  other  files.  Buffers  are  only  allocated  to  files  that  are 
actively  in  use  (i.e.  "open"). 

Allocation  Strategy  Nodules  (ASH) 

The  Allocation  Strategy  Modules  keep  track  cf  the 
available  records  on  a  device.  They  are  responsible  for 
allocating  records  for  a  file  that  is  being  created  or 
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expanded,  and  deallocating  records  for  a  file  that  is  being 
erased  or  truncated.  The  POSE  requests  that  a  record  be 
allocated  when  needed,  the  ASH  actually  selects  the  record. 

Quite  frequently,  the  ASH  functions  are  incorporated 
into  either  the  rosR  or  Osh.  Xn  this  paper  those  functions 
will  be  kept  as  separate  as  possible  by  explicitly 
recoqnixing  the  separate  ASR  level. 

Device  Strategy  nodules  <D5E) 

When  a  large  portion  of  a  file  is  to  be  read  or 
written,  aany  records  aust  be  processed.  The  Dewice  Strategy 
Module  considers  the  device  characteristics  such  as  latency 
and  access  tine  to  produce  an  optiaal  I/O  sequence  frca  the 
POSE  and  ASR  requests. 

Input/output  Control  Systee  (IOCS) 

The  Input/Output  Control  System  coordinates  all 
physical  I/O  on  the  computer.  Status  of  all  outstanding  I/O 
in  process  is  aaintained,  new  I/O  requests  ace  issued 
directly  if  the  device  and  channel  are  available,  otherwise 
the  request  is  queued  and  autoaat  ically  issued  as  soon  as 
possible.  Automatic  error  recovery  is  attempted  when 
possible.  Interrupts  from  devices  and  unrecoverable  error 
conditions  are  directed  to  the  appropriate  routine.  Alaost 
all  «odern  operating  systems  have  an  IOCS. 
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rilfi  systeas  versus  Data  Hanageaent  Systeas 

In  the  literature  there  is  often  confusion  between 
systeas  aa  described  above,  which  this  paper  calls  «file 
systeasM  and  systeas  vlich  will  le  called  "data  aanageaeut 
systees",  such  as  DB-KDiion  67>,  GIR-1<N«1  67>,  and 
TDBSCBlei  67>.  The  confusion  is  to  to  expected  since  both 
types  of  systeas  contain  ail  of  the  functional  levels 
described  above.  The  systeas  differ  primarily  cn  the 
eaphasis  placed  on  certain  levels. 

In  general  file  systeas,  the  file  is  considered  the 
aost  iaportaat  itei  and  eaphasis  is  placed  on  the  directory 
organization  (Logical  rile  Systea)  and  the  lover 
hierarchical  levels.  It  is  expected  that  specialized  access 
aethods  uill  be  written  by  users  or  sipplied  with  the  systea 
as  needed. 

In  aost  data  aanageaent  systeas,  the  individual  data 
iteas  are  considered  the  aost  iaportaat  aspect,  therefore 
eaphasis  is  placed  on  elaborate  access  aethods  with  ainiaal 
eaphasis  on  the  lover  levels  of  abstraction.  Because  of  the 
heavy  eaphasis  on  a  single  level,  data  aanageaent  systeas 
tend  tc  appear  less  hierarchical  than  file  systeas  since  the 
lover  levels  are  often  absorbed  into  the  access  aethods. 
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The  virtual  aeaory  interface  provided  by  tbe  Logical 
Pile  Systnp  allows  for  very  flexible  user  applications  and 
access  methods.  In  a  PL/1-Hke  notation,  calls  tC  the 
Loqicai  Pile  system  are  of  the  fora: 

IPS,,.  Read/ Write  (Pilenaae,  Addr  1,  Addr  2,  Number)  ; 
where  Addrl  is  the  aain  storage  address,  Addc2  is  the  file 
virtual  aeaory  address,  and  Number  is  the  nuaber  of  eleaents 
to  be  moved. 

In  this  paper  eleaents  will  be  assuaed  to  be  8-bit 
hytes.  For  exaeple,  a  request  to  read  100  bytes  frop 
location  200  vithin  the  file  named  ALPHA  into  main  storage 
location  1214  could  be  expressed: 

LFS_Bead  ('ALPHA*  ,  1234,  200,  100); 

Sequential  fixed-length  records,  sequential 

variable-length  records,  and  direct-access  fixed-length 
records  are  common  access  aethods.  All  cf  these 
organizations  and  aany  sore  can  be  realized  using  a  file's 
virtual  meaory.  Note  that  the  records  processed  by  the 
access  aethods  are  "software”  records  and  have  no  relation 
to  the  ph ysical /loqica  1  records  processed  by  the  POSH  and 
D  511  . 


accf.ss  methods  mocil  «9 

Sequential  and  Direct-Access  Fixed-Length  Record  Access 
Met  hods 

to  simulate  those  acctti  methods,  the  file's  virtual 
memory  is  treated  as  a  sequence  of  records  of  the  desired 
length,  l. 

To  access  these  records  sequentially,  a  position 
counter,  PC,  is  set  aside  that  starts  at  0  and  is 
incremented  by  L  after  each  read  or  write.  The  position 
counter  therefore  finds  the  location  of  the  next  sequential 
record.  The  routine  could  be  written  as: 

Lrs_Read  (Filename ,  location,  PC,  1); 

PC  «  PC  ♦  L; 

To  access  these  records  by  dir ect-access  there  is  no 
need  for  a  position  counter  since  the  desired  record,  r,  can 
be  found  at  location  (c-1)*L  in  the  file's  virtual  aeaory. 

i 

This  routine  could  be  written  as: 

LFS_Read  (Filename,  Location,  i)  ; 

Sequential  Variable-length  Record  Access  Method 

The  Sequential  Variable-Length  Record  Access  Method 
treats  the  file  as  an  ordered  sequence  of  records,  each 
record  may  be  a  different  longth.  This  method  can  be 
implemented  by  preceeding  each  record  with  a  "hidden"  length 
field. 

These  records  can  be  accessed  using  a  variation  cf  the 
Sequential  Fixed-Length  scheme.  For  example: 
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Layout  of  Virtual  Hccory  for  Variable-Leugth 
Record  Access  Method 
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LPS_Fead  (Filename,  L,  PC,  4);  /♦  Get  4  byte 

length  */ 

LPS_aead  (F ilena me ,  Location,  PC*4,  t)  ;  /•  Get 

data  */ 

PC  =  PC  ♦  L  ♦  4;  /*  Update  position  counter  V 

Other  Access  Methods 

The  above  examples  were  presented  to  illustrate  the 
ease  with  which  conventional  access  Methods  can  be  supported 
under  this  file  systen  design.  The  real  importance  of  the 
virtual  memory  concept  is  not  its  ability  to  provide 
traditional  access  Methods,  but  the  ease  and  flexibility 
with  which  problem-oriented  access  Methods  can  be  developed. 
The  programmer  is  able  to  design  access  Methods  based  cn  the 
needs  of  his  problea  rather  than  forcing  his  problem 
solution  to  be  constrained  by  a  siall  set  of  limited  access 
methods.  For  example,  NelsonCNel  65>  discusses  soie  flexible 
and  conplex  file  structures  that  can  he  used  "as  an  adjunct 
to  creativity". 

The  power  of  a  computer  reaches  its  peak  when  it  is 
capable  of  amplifying  the  creativity  of  the  programmer.  A 
systen  that  restricts  the  programmer's  ability  to  express 
his  ideas  provides  hia  questionable  service. 
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A  user's  program  references  each  file  by  Means  of  a 
unique  symbolic  bate,  jit  is  the  function  of  the  Logical  File 
System  to  convert  the  symbolic  Rase  reference  into  its 
corresponding  unique  file  identifier.  The  Logical  File 
system  performs  the  mapping  using  a  "file  directory 
organiza  ti  on". 

In  the  simplest  case  the  file  directory  is  entirely 
stored  in  main  storage  as  a  two-entry  table.  The  two  entries 
are  the  syabolic  file  name  and  its  corresponding  file 
identifier.  A  look-up  routine  is  all  that  is  needed  to  serve 
the  function  of  the  Logical  rile  Sy ate  a.  This  approach  is 
used  by  several  file  systeus  because  of  its  simplicity  and 
efficiency.  Unfortunately,  the  nuaber  of  files  that  are 
allotted  in  the  file  systea  is  restricted  by  the  amount  of 
aain  storage  available  for  the  file  directory. 

To  remove  the  above  limitation,  many  file  systems  keep 
the  file  directory  on  secondary  storage.  The  file  directory 
can  be  treated  as  a  standard  file  if  its  file  descriptor  is 
always  known.  This  allows  the  file  directory  to  be 
processed,  expanded,  and  truncated  using  the  normal  file 
system  mechanisms.  The  Logical  File  System  mapping  still 
involves  a  table  look-up,  only  this  time  the  table  is 
contained  in  a  file's  virtual  memory  rather  than  main 
storage.  The  calls  to  the  Basic  File  Systea  are  essentially 
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the  same  as  the  calls  to  the  Logical  Pii.c  System,  only  a 
file  itientifiet  is  specified  rather  than  a  syabslic  file 

name . 

A  few  of  the  advanced  file  systems  have  introduced  the 
concept  of  the  hierarchical  file  directory.  Fro*  a  si»pie 
point  of  view,  a  file  directory  hierarchy  resembles  and 
serves  a  similar  purpose  to  a  PL/1  data  structure.  In 
practice,  certain  files  are  classified  as  "directories"  in 
addition  to  their  norial  attributes.  The  earlier  model  of 
the  Logical  File  System  implied  that  there  was  only  one 
directory  file.  This  file  contained  the  file  identifiers  for 
all  the  other  files,  called  "data  files".  This  has  been 
extended  to  allow  the  base  directory,  often  called  the  "root 
directory",  to  contain  file  identifers  for  directory  files 
as  well  as  data  files.  Each  subsequent  directory  file  can 
contain  file  identifers  for  ether  directory  files  as  well  as 
data  files. 

Figure  3.8  illustrates  a  file  directory  hierarchy.  The 
files  ft,  B,  C,  and  D  are  directory  files,  all  the  others  are 
data  files.  The  data  files,  as  well  as  directory  files,  do 
not  necessarily  have  unique  symbolic  names.  There  are  3  data 
files  in  Figure  3.8  named  "X",  as  in  Pl/1  this  ambiguity  is 
solved  by  using  qualified  names  such  as  "A.X",  "ft.B.D.X", 
and  "A.C.X". 

The  file  directory  hierarchy  serves  many  purposes  in 
adniticn  to  providing  flexible  and  versatile  facilities  for 
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praqramaer  usage.  "File  sharing"  and  "controlled  access" 
among  users  are  reij  closely  tied  to  the  hierarchical 
directories.  Certain  of  these  features  are  discussed  in  the 
paper  by  Daley  and  Meumann<Daley  6S>.  A  aore  detailed 
trea  taent  of  this  topic  Mill  be  presented  in  a  subsequent 
paper  by  this  author. 

The  implementation  of  the  Logical  File  System  for  a 
file  directory  hierarchy  is  a  simple  extension  of  the  single 
directory  technique.  After  finding  the  correct  file 
identifier  in  the  root  directory*  it  is  either  the  data  file 
desired  or,  if  a  secondary  directory  file,  is  used  in 
exactly  the  same  manner  as  the  root  directory  identifier  to 
advance  one  more  level  in  the  hierarchy. 
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As  explained  in  the  Overview  section,  a  file  is 
physically  located  otr  secondary  storage  as  an  ordered 
collection  of  distinct  records,  the  inforaation  that 
describes  a  file's  size,  access  rights,  device  address  or 
addresses,  and  the  sapping  algorithm  must  be  saintained  by 
the  file  aystea. 

In  a  sisple  file  systes  this  inforaation  can  be 
incorporated  into  the  file  directory  as  long  as  there  is  a 
unique  one-to-one  tapping  of  file  nase  onto  tile.  In  a 
sophisticated  file  systes  with  features  such  as  (1) 
hierarchical  file  directory,  (2)  aliases  that  allow  a  single 
file  to  be  referenced  by  different  nases,  (3)  links  that 
allot*  a  file  to  be  referenced  froa  various  directories  is 
the  file  hierarchy  or  froa  different  users,  and  (4) 
removable  or  detachable  "voluaes"  or  devices,  the  unique 
aapping  cannot  be  guaranteed. 

To  produce  an  unambiguous  file  systea,  the  file 
directory  inforaation  is  divided  into  three  parts,  the  file 
naae,  identifier  and  the  descriptor.  The  file  naae 
directories  are  the  aappings  between  a  syabolic  file  naae 
and  the  corresponding  identifer.  The  precise  loeatiens  cf 
the  file  descriptors  can  differ  for  different 
iapleaentaticns,  but  uniquely  defined  by  the  identifer.  In 
fact,  since  the  file  descriptors  usually  need  not  be 
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soarchod,  they  need  not  be  contiguous.  Usually  they  ate 

collected  in  cither  (1)  a  special  systea  vide  file,  (2)  a 
collection  of  files*  each  located  or  a  separate  device  oc 
voluae,  or  (3)  bidden  within  the  syabolic  file  naae 
d. rector ica. 

Although  it  is  usually  not  possible  to  keep  the 

syabolic  file  directories  in  aain  storage,  the  nuaber  of 
files  actively  in  use  is  sufficiently  saall  that  the 

corresponding  file  descriptors  can  be  placed  in  a 
core-resident  table  called  the  Active  Pile  Directory  or  Open 
Pile  Directory. 

It  is  the  function  of  the  Basic  Pile  Systea  to  use  the 
unique  file  identifier  to  locate  the  file  descriptor  and 
place  it  in  the  Active  Pile  Directory  unless  it  has  already 
been  "opened**.  The  Basic  Pile  Systea  also  checks  that  the 
action  requested  upon  the  file  such  as  read,  write,  cr 

delete  does  not  violate  the  restrictions  specified  in  the 
file  descriptor. 


After  verifying 

legal  access 

to  the  file,  the 

Basic 

Pile  Systea 

passes 

control  to 

the  appropriate 

File 

Or  gan  izat ion 

stra  tegy 

Module  as 

specified  in  the 

file 

descriptor  entry. 
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The  priaary  function  of  the  File  Organization  Strategy 
nodule  is  to  aap  a  file's  virtual  aeaory  address  ante  a 
corresponding  physical  record  nusfccr.  There  are  at  least 
three  coaaon  physical  file  organization  strategies: 
sequential,  linked,  and  indexed. 

Sequential  File  Organization  Strategy 

The  Sequential  File  Organization  Strategy  is  used  by 
■  ost  of  the  older,  sispler,  and  non-dynanic  file  systeas. 
Under  this  technique  logically  consecutive  records  are 
physically  consecutive.  For  exaaple,  if  each  record  is  1000 
bytes  long,  virtual  address  3214  would  be  located  in  the 
fourth  logical  record.  It  the  first  logical  record  (i.e., 
the  one  containing  virtual  address  0)  is  physical  record 
120,  the  record  containing  victual  address  3214  would  be 
physical  record  123. 

There  are  two  notable  advantages  claiaed  fer  this 
technique.  Firstly,  the  napping  is  very  sitple  and 
efficient.  The  only  information  needed  is  the  fixed  record 
size  and  the  address  of  the  first  record,  secondly,  if  the 
file  is  to  be  processed  in  a  sequential  aanner,  the 
consecutive  organization  allows  for  ainiaizinq  device 
latency  and  access  tine. 

Although  the  first  point  is  indisputable,  the  second 
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claiaed  advantage  is  open  to  question.  If  there  is  sore 
than  one  file  on  the  sue  device  that  is  actively  is  use,  as 
is  ccaion  is  a  tulfci- 1«  sjking  ervirocsent,  then  the  device 
read /write  positioning  will  be  switching  rapidly  along  the 
active  files,  defeating  the  asnuaed  sequential  accessing, 

the  aa jot  disadvantage  of  this  sequential  organisation 
is  that  the  aaxiaua  size  of  the  file  mat  be  assuaed 
statically  before  creating  the  file.  By  specifying  too 
snail  a  size,  the  task  will  te  forced  to  terninate  if  aore 
space  is  needed.  If  too  large  a  size  is  assused,  as  is 
coaeon,  there  is  aueh  wasted  space  and  fragaentat icn . 

This  technique  aa y  be  reccaaended  for  single-tasking 
systcis  with  few  peraanent  files  and  very  few  files 
s iau It aneously  in  use.  It  light  be  useful  for  a  large 
information  utility  systei  which  is  based  on  a  large  nuaber 
of  independent,  low  cost,  low  usage,  high  capacity  devices 
such  as  data  cells  where  wasted  space  is  not  a  significant 
o  rob  lea, 

linkvd  File  Organization  Strategy 

The  Linked  and  Indexed  File  Organization  Strategies 
allow  for  files  to  dynaaically  grow  and  shrink.  The  linked 
technique  was  probably  developed  first  since  it  is  sivpler 
and  caphasizes  sequential  characteristics  which  were 
priaarily  used  in  early  file  systers. 

The  linked  organization  requires  each  record  of  a  file 
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to  specify  the  location  of  the  next  logical  record, 
analogous  to  the  "links"  on  a  chain.  The  file  descriptor 
specifies  only  the  location  of  the  first  record.  It  tells 
nothing  about  the  locations  of  the  other  records.  As  the 
file  grows,  new  records  are  dynaaically  allocated  and  linked 
onto  the  file. 

for  sequentially  processed  files,  the  linked  technique 
provides  a  very  siaple  and  efficient  Mechanise.  A  few  bytes 
are  used  in  each  rocord  to  record  the  link,  and  since  record 
sizes  are  usually  in  the  range  of  1G00  bytes  the  overhead  is 
siniaal.  Unfortunately,  candon  or  direct-access  file  usage 
poses  serious  problems.  If,  for  example,  the  last  access 
was  to  a  data  area  in  logical  record  5,  a  reference  to  an 
area  in  logical  record  15  will  require  9  intermediate  I/O 
accesses  to  find  the  links  before  reaching  the  desired 
record.  The  Linked  File  Organization  Strategy  has  been  used 
satisfactorily  on  systems  where  the  vast  Majority  of  files 
are  accessed  sequentially. 

Indexed  File  Organization  Strategy 

The  Indexed  File  Organization  Strategy  is  a 
significant  variation  to  the  linked  technique.  Records  are 
dynaaically  allocated  as  needed,  but  rather  than 
distributing  the  record  addresses  throughout  the  file  as 
links,  they  are  collected  together  as  a  table.  The  logical 
record  nuiber  is  used  as  an  "index"  in  the  table  to  find  the 
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corresponding  physical  record  nueber. 

If  files  are  lisited  tc  stall  or  eedi.u»  sizes,  the 
ind*»<  table  can  be  stored  as  pact  of  the  file  descriptor. 
It  files  are  allowed  to  be  arbitrarily  large,  the  index 
table  nust  itself  be  treated  as  a  file  and  is  broken  into 
separate  records.  In  the  foraer  case,  sequential  and  randoa 
access  processing  proceed  easily  and  efficiently.  Is  the 
latter  case,  sequential  processing  is  very  efficient,  except 
for  intermittent  accesses  for  the  next  portion  of  the  index 
tabic.  Bandon  processing  aay  be  very  efficient  if  localized 
to  a  sitple  index  table  block;  in  any  case  it  will  newer 
exceed  a  saall  nurber  of  interaediate  accesses,  usually  one 
or  two,  for  totally  randoa  processing. 

The  Indexed  File  Organixation  Strategy  has  the 
advantage  of  allowing  the  concept  of  a  "sparsely  filled" 
file.  If  we  assume  that  each  physical  record  is  1000  bytes- 
and  each  index  (record  nuaber)  is  4  bytes,  then  the  index 
table  for  a  file  that  is  250,000  bytes  long  would  require 
250  indexes  or  1000  bytes.  Ey  designating  a  special  cede, 
such  as  0,  to  indicate  an  index  for  a  non-allocated  reccrd, 
a  file  can  be  created  with  specific  contents  at  locations 
10,000,  40,000,  and  247,000  but  with  unspecified  contents 
elsewhere.  By  convention,  unspecified  contents  are  usually 
initialised  as  zero  by  the  file  systea.  The  above  sparse 
file  would  only  require  four  physical  records,  three  records 
for  the  specified  portions  of  the  file  and  one  record  for 
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the  index  table.  As  tore  information  is  written  intc  a 
sparse  file,  aore  physical  records  will  be  allocated  as 
needed. 

The  indexed  organisation  provides  a  simple  and 
efficient  way  to  use  programming  techniques,  such  as  "hash 
coding**  or  "random  entry**  tables,  that  require  a  large 
though  sparse  virtual  Memory. 

Many  of  the  most  recent  file  systems  have  adopted 
techniques  similar  to  the  Indexed  File  Organization. 
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Ailocatign  strategy  Hod  ul«s 

When  the  POSH  saps  a  valid  write  request  onto  a  logical 
record  for  which  a  physical  record  has  not  been  allocated, 
the  ASH  is  called  to  find  an  available  record  for  use.  There 
are  two  coaaon  techniques  used  to  keep  track  of  available 
records.  The  first  technique  links  all  available  reccrds 
together.  This  aethod  is  often  used  in  conjunction  with  a 
Linked  pile  organization  Strategy  Nodule.  The  second 
technique  uses  a  "bit  nap"  for  each  device.  A  bit  aap  is  a 
function  which  operates  on  a  bit  string  and  describes  the 
relationship  between  a  bit  position  and  a  physical  record  on 
the  device.  For  exaaple  a  convenient  bit  aap  sight  be:  bit 
0  corresponds  to  physical  record  0,  bit  1  to  physical  record 
1,  etc.  If  a  bit  is  set  to  0,  the  corresponding  recced  is 
available  for  allocation,  otherwise  it  has  already  been 
allocated  to  a  file.  The  tit  aap  provides  a  very  ccapact 
representation  cf  the  allocation  inforaaticn.  The 
allocation  states  cf  a  device  with  a  capacity  of  8,GCC,0Q0 
bytes  divided  into  80G0  1000-byte  records  can  be  stored  in  a 
in'!  byte  bit  cap.  In  a  file  systea  with  a  large  nuober  of 
high-capacity  direct-access  devices,  it  say  be  iapossible  to 
keep  all  the  bit  naps  in  sain  storage.  The  bit  aap  aay  be 
subdivided  into  sections,  such  as  a  separate  bit  nap  for 
each  group  of  800  records.  Only  one  section  of  the  bit  aaF 
for  a  device  is  kept  in  sain  storage  at  a  tiae,  the 
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regaining  sections  are  left  stored  on  the  device. 

Since  sequential  processing  is  a  very  coaaon  file 
usage,  the  ASM  say  atteapt  to  allocate  records  tc  take 
advantage  or  this  fact,  of  course,  any  specific  File 
Organization  strategy  Module  and  Device  Strategy  Module 
group  are  expected  to  be  cooperative  with  the  Allocation 
Strategy  Modules  tc  optimize  overall  performance.  The 
precise  nature  cf  eeaningful  cooperation  vould  be  too 
detailed  to  discuss  in  this  paper. 
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Device  Strat eav  Bcdules 

Its  addition  to  the  obvious  "read"  and  "write" 
functions,  direct  access  devices  often  require  additional 
I/O  eoBfiands,  such  as  MseeX"  and  "search",  for  proper 
positioning.  The  POSH  and  ASft  deal  only  with  the  logical  act 
of  of  reading  and  writing.  They  transfer  a  set  cf  requests 
to  the  DSH  of  the  fora:  "read  record  24  into  location  5400, 
read  record  49  into  location  6400,  and  write  record  27  froi 
location  9324'*.  The  DSH  Bust  translate  these  requests  into 
the  obscure  I/O  list  foraat  required  for  the  particular 
device. 

Furthermore,  due  to  the  device  characteristics  such  as 
latency  and  access  tiwe,  the  order  in  which  the  requests  are 
perforsed  affects  the  total  aaount  of  tiae  that  the  device 
is  kept  "busy".  For  exaaple,  if  records  24  and  27  are 
"closer",  in  soae  sense,  to  each  other  than  record  49,  it 
aight  be  gore  efficient  to  read  record  24,  write  record  27, 
and  then  position  to  read  record  49. 
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ImL'fil&fti  sjstea 

The  Input/Output  Control  Syntes  coordinates  all  the 
physical  I/O  on  the  computer*  On  nost  nodera  conputers  there 
are  conplex  interdependencies  aaong  the  physically 
independent  I/O  devices.  Usually  this  dependency  occurs  due 
to  the  dedicated  nature  of  "selector”  channels  and  device 
control  units  that  can  switch  to  any  device  but  can  cnly 
service  one  device  at  a  tine.  For  very  high-speed  devices, 
such  as  druns,  the  sain  storage  access  tine  can  be  an 
iaportant  factor.  If  toe  nany  sinultaneous  nenory  requests 
occur,  "overrun"  can  occur  resulting  in  erroneous  data 
transnission .  The  IOCS  keeps  track  of  the  status  of  all 
devices,  control  units,  and  channels*  When  an  I/O  operation 
is  requested,  the  IOCS  checks  to  insure  a  clear  path  tc  the 
device  through  the  channels  and  control  units  and  that  no 
I/O  capacity  Units  will  be  exceeded.  If  if  is  not  possible 
to  issue  the  requested  I/O  operation,  the  IOCS  stores  the 
request  on  a  queue.  The  I/O  will  be  issued  at  a  later  tine 
when  all  conditions  are  satisfied.  Since  the  I/O 
interdependencies  nay  exist  anong  all  devices,  every  I/O 
operation  whether  for  the  file  systen  or  dedicated  special 
purpose  device  nust  be  funnelled  through  the  IOCS. 

Although  aost  nodern  I/O  devices  are  very  reliable, 
spurious  errors  do  occur.  Usually  the  retry  cr  recovery 
procedure  is  very  siaple,  in  such  a  case  the  IOCS  will 
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attempt  corrective  Measures* 

The  caller  tc  the  IOCS  is  informed  of  the  status  of  his 
I/O  request,  for  exaaple  (1)  successful  cospletion,  (i) 
unrecoverable  error  condition,  or  (3)  asynchronous 
interrupt. 

The  sophistication  and  scope  of  the  IOCS  depends  upon 
the  devices  to  he  handled  and  the  goals  of  the  file  systea 
and  operating  systea. 
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CHAPTER  POtlB 

Hu  It i-Coapu ter  network  Bovironnent 


Basest  21LDi 

A  general  file  systea  design  nodel  aust,  of  course,  be 
sodified  and  elaborated  to  satisfy  the  needs  of  any  specific 
desired  file  systea  onvironncat.  To  illustrate  the 
refineaent  process,  a  unigue  file  systee  design  will  be 
presented  for  a  aulti-coeputer  network. 

Hulti-coiputer  networks  are  becoeing  an  increasingly 
important  area  of  coaputer  tech  nolog y<H ad  68> .  There  are 
several  significant  reasons  behind  the  growth  of 
aulti-coaputer  networks: 

1.  To  increase  the  power  of  a  coaputer  installation 

In  a  nodular  aanner,  especially  if  (a)  it  is  not 
possible  to  acquire  a  larger  processor,  (b) 

reliability  is  iaportant,  or  ( c)  there  are 
real-tine  or  tine-sharing  constraints. 

2.  To  serve  the  co-ordination  requinnents  of  a 
network  of  regional  coaputer  centers. 

3.  To  support  the  accessibility  to  a  nation-wide  data 


base 
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An  vi  a  •  p  1  *>  of  the  environsont  to  be  considered  for  this 
paper  can  be  illustrated  in  figure  4,1*  This  type  of 
sulti-cosputer  network  has  been  in  lisited  use  for  several 
years  in  aaay  configurations.  The  IBtt  7094/7044 
Si  rcct-eeuplcd  syst«a<S©seR  69>  was  probably  one  cf  the 
e«rlie«^  practical  exasples  of  such  an  Inter-connected 
arra  nqcsent. 

There  are  several  isplicit  constraints  ispoaed  upon  the 
■u It i- computer  systes  illustrated  in  figure  4.1: 

1.  Independence  of  Central  Processors. 

Each  of  the  central  processors  operate  independently 
such  that  there  are  no  direct  processor-to-processor 
data  transfer  nor  signaling;  and  furtheracre  there 
is  no  "tester H  processor. 

2.  hon-shared  Hesory. 

Each  central  prccessor  has  its  own  sain  storage 
unit.  These  units  are  not  shared  with  nor  accessed 
by  another  central  processor. 

3.  Inter-locked  Device  controller: 

The  device  controllers  act  as  "traffic  cops"  to  the 
actual  I/O  direct  access  devices.  They  control  the 
traffic  between  a  cosputer's  I/C  channel  and  a 
selected  I/O  device,  k  single  device  controller  will 
only  accept  requests  fros  one  channel  at  a  tise  and 
will  only  select  one  I/O  device  (asong  those  under 
its  control)  at  a  tise.  once  a  device  controller 
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Exaeple  of  8ulti-co»puter  File  Syste*  Network 
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connects  a  channel  with  *  device,  the  connection 
retains  intact  until  the  channel  releases  the  device 
or  an  I/O  error  occur*. 

Tb«*  envircnient  described  above,  although  veil  within 
the  boundaries  of  currant  technology,  has  not  been  the 
subject  of  auch  investigation.  Such  ccnf  igurationa  ace 
presently  very  eipensive  and.  therefore,  chosen  cnly  for 
very  specialized  situations.  Even  then  there  are  only  tec  or 
three  processors  and  very  specialized  software  and 
operational  factors.  A  discussion  of  the  CP-67/CBS  Tine 
Sharing  Systea  <18  tt  68aXSea  68>  will  serve  to  establish  the 
relevance  of  the  aulti-ccepu ter  network  environment. 

The  CP- 67/CM S  Tine  Sharing  Systea  uses  the  special 
hardware  features  of  a  single  IBM  Systen/360  nodel  67 
processor  augaented  by  software  to  produce  an  apparant 
envitonaent  corresponding  to  the  aulti-coaputer  network 
illustrated  in  Figure  t»,  1,  with  aany  independent  central 
processors,  device  controllers,  and  direct  access  I/O 
devices.  In  practice  a  typical  single  processor  36C/67 
configuration  would  produce  the  affect  of  about  30  active 
processors  ("virtual**  Systea/360  aodel  65  processors  each 
with  a  256, CCC  byte  aeaocy}  and  50  active  device 
controllers.  More  detailed  descriptions  of  the  CF-67/CMS 
Systea  can  be  found  in  the  References.  In  the  traditional 
sense  of  tine-sharing,  each  user  of  the  CP-67/CMS  Systea  is 
provided  with  a  "virtual**  coaputer  operated  froa  a  ciaulated 
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operator  console  (actually  an  augmented  reaote  terainal). 
Rost  iiportantly,  each  "virtual"  computer  (i.e.  usei ) 
operates  logically  independently  of  all  other  "virtual** 
coaputers  except  for  the  specified  inter-connected  l/o 
devices  and  device  controllers. 
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EE2l>ifIs  LEising  Id  Hulti-Coaputer  Networks 

There  are  uny  probleas  associated  with  the 
a ui t. i-coaputer  file  systea  network*  Soae  of  these  probleas 
are  unique  to  this  eRvironnent.  Other  probleas  have  been 
solved  in  traditional  fiJe  systeasCCorb  62><Salt  6SXScie 
but  the  solutions  require  aajor  revisions  due  tc  the 
peculiarities  of  the  environment.  The  aost  significant 
protleas  are  listed  briefly  below. 

1.  No  shared  teiory. 

Usually  file  systems  co-ordinate  the  status  of  the 
files  and  devices  by  using  tain  storage  accessable 
tables  and  data  areas  that  describe  file  status, 
access  rights,  interlocks,  and  allocation.  There  is 
no  such  cocaon  coaaunicat ion  area  in  sain  storage 
that  can  be  accessed  by  all  the  independent 
processors. 

2.  No  inter-ccaputer  coaaunica tion. 

Nulti-coaputer  configurations  usually  provide  a 
mechanism  for  sending  signals  or  data  transfers 
between  the  separate  processors.  With  this 
capability  the  non-sharcd  neaory  problea  cculd  be 
solved  by  either  (a)  electing  one  processor  tc  be 
the  "laster"  processor  that  coordinates  the  ether 
processors,  or  (b)  supply  all  the  processors  with 
enough  irferaation  such  that  each  processor  knows 
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what  all  the  othdr  processors  aro  doing*  The  concept 
of  a  "taster"  processor  opposes  the  intecded 
homogeneous,  independent  processor  assuaption ,  The 
possibility  of  supplying  status  information  to  all 
other  processors,  although  reasonable  for  a  three  or 
four  processor  configuration,  vas  not  considered  a 
feasible  solution  for  a  system  with  hundreds  of 
processors  and  devices  and  thousands  of  files.  For 
these  reasons,  inter-computer  communication, 
although  as  available  capability,  vas  not  included 
as  a  required  capability  of  the  nulti-conputer 
environment  described  above. 

3.  No  pre-arranged  allocations. 

For  snail  specialised  multi-computer  file  netvorks, 
each  processor  can  be  "assigned"  a  specific  area  of 
a  device  or  set  of  devices  that  can  be  used  tc  vrite 
new  files,  all  ether  processors  can  only  read  froa 
this  area  by  convention.  This  prevents  the  danger  of 
tvo  independent  processors  vriting  files  at  the  same 
place.  Such  an  "arrangement"  is  not  practical  for  a 
large,  flexible  multi-computer  file  netuork  since 
the  static  assignment  of  secondary  storage  space 
does  not  take  account  of  the  dynamic  and 
unpredictable  requirements  of  the  independent 
processors. 

U.  Extendable  device  and  file  allocation. 
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The  nuaber  of  devices  and  sizes  of  devices  as  veil 
as  the  nuaber  and  sizes  of  files  are,  within  reason, 
unliaited.  For  etaaple,  a  specific  aaouvt  cf 
secondary  storage  equivalent  to  ICO,  000  card  iaages 
could  be  used  to  hold  10  files  of  10,000  card  each 
or  1,000  files  of  100  cards  each.  This  consideration 
discourages  techniques  that  result  in  a  strong 
efficiency  or  eain  storage  capacity  dependency  on 
the  Hsize  and  shape*1  of  the  file  systei.  Of  course, 
the  Magnitude  of  the  file  systea  size  will  affect 
the  operation,  but  arbitrary  restrictions  such  as 
"no  aore  than  64  files  on  a  device"  would  be 
discouraged  unless  essential. 

5.  Removable  voluaes. 

It  has  becoae  coaaon  to  differentiate  between  the 
I/O  aechanisa  used  to  record  or  read  inforaation, 
called  a  "device",  and  the  physical  aediua  on  which 
the  inforaation  is  stored,  called  a  "volute".  For 
aost  druas  and  aany  disk  units,  the  device  and 
voluae  are  inseparable.  But,  for  Magnetic  tape  units 
and  aany  cf  the  saaller  disk  units  the  veluae, 
aagnetic  tape  reel  and  disk  pack  respectively,  are 
renovable.  It  is  intended  that  the  file  systea 
include  files  that  are  on  unaounted  voluaes 
{disconnected  froa  an  I/O  device)  as  well  as  Mounted 
voluaes.  Therefore,  a  configuration  that  consists  of 
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ten  disk  units  nay  hate  a  file  systea  that 
encompasses  hundreds  cf  volumes,  only  ten  of  which 
may  be  actively  in  use  at  a  tiae.  Since  removing  and 
mounting  a  voluae  takes  several  ainutes  of  aanual 
effort,  it  Kill  he  assumed  that  the  "working  set"  of 
voluaes  (volutes  that  contain  files  that  are 
actively  in  use)  retains  static  for  reascnable 
periods  of  tiae  and  is  less  than  or  equal  tc  the 
nuaber  of  devices  available.  The  fact  that  voluaes 
are  reaovable  and  interchangeable  (i.e.  aay  be 
aounted  cn  different  devices  at  different  tines) 
does  affect  the  organization  of  the  file  systea,  For 
exaiple,  a  scheae  that  involved  linking  files 
together  by  aeans  cf  pointers  (chained  addressing) 
could  require  aounting  voluaes  just  to  continue  the 
path  of  the  chain  even  though  little  or  no  "logical" 
inforaation  vas  requested  froa  files  on  that  voluae. 
In  the  worst  case,  it  sight  be  necessary  to  aount 
and  unaount  all  the  voluaes  of  the  file  systea  to 
locate  a  desired  file.  Such  a  situation  should 
definitely  be  avoided  if  not  totally  eliminated  by 
the  file  systea . 

6.  Structured  file  directories  and  file  sharing. 

In  a  traditional  file  systea,  the  sapping  betveen 
the  symbolic  file  naae  and  the  corresponding  tile 
vas  accocplished  by  aeans  of  a  single  Master  File 
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Directory.  For  modern  file  systems  with  thousands  of 
files  scattered  over  hundreds  of  volumes.  it  became 
desirable,  if  not.  necessary,  to  fore  groupings  of 
files  by  aeans  of  Secondary  Pile  Director ies<Daley 
6b>.  These  groupings  are  often  used  by  the  systai  to 
associate  users  with  files  they  own  (User  File 
Directories).  This  capability  is  also  available  to 
the  user  to  arrange  his  files  into  further 
sub-groups  (libraries)  or  into  separate 
project-related  groupings.  Occasionally  it  becomes 
necessary  for  a  file  to  be  included  in  tec  cr  eore 
groupings  (e.g.  accessible  by  nore  than  cue  User 
File  Directory)  with  potentially  different  access 
privileges  (protection)  associated  uith  each 
grouping.  Many  of  these  features  that  are  relatively 
easy  to  implement  in  a  traditional  file  system  are 
complicated  by  the  introduction  of  independent 
processors  and  reaovable  voluaes. 

7.  Pail-safe  operation. 

Reliable  operation  is  a  very  important  reguireient 
of  a  general  purpose  file  system.  There  are  many 
known  techniques  for  I/O  error  and  systematic  backup 
and  salvage  pioceduzes  that  are  applicable  to  this 
environment.  The  important  problem  associated  with 
the  aulti-coaputer  network  is  that  potential  error 
conditions  exist  that  are  not  normally  fcund  in 
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ttaditlonal  single  coaputei  file  systeas.  for  a 
single  coaputer  systea,  a  processor  error  (including 
unexpected  processor  disconnection,  i.e.  "turning 
off")  is  a  rare  occurrence.  Such  a  t;ituaticn  is 
remedied  by  repairing  whatever  physical  hardware  is 
necessary  and  then  running  a  special  "salvager" 
prograa  tc  bring  the  file  systea  into  a  well-defined 
operational  state*  In  the  environaant  of  a 
■ulti-coeputer  network,  processors  nay  be  connected 
or  disconnected  at  any  tine  without  any  awareness  by 
the  other  processors.  To  prevent  any  inconsistent 
file  systei  operation  by  the  other  processors  and 
eliminate  the  need  for  usually  tine-ccnsuiing 
salvage  techniques,  it  is  necessary  to  keep  the  file 
syste*  in  a  well-defined  consistent  state  at  all 
tises. 
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*  £il«  Sjtstes  Jegiaa 

The  purpose  of  the  ceaainder  of  this  paper  is  tc  «ffly 
the  organisation  presented  in  the  Pile  System  Design  Model 
section  to  solve  the  prcblcac  associated  *  it  b  a 
■ult i-cosputer  file  systea  network.  Discussion  of  the  Access 
Methods  and  Input/output  Ccatcol  Systea  will  be  oaitted. 
This  is  necessitated  for  brevity  and  consideration  of  the 
facts  that  the  Access  Methods  are  highly  application 
oriented,  as  discussed  in  a  previous  section,  and  that  the 
Input/Cutput  Control  Systea  is  usually  a  basic  and  cctacn 
coaponent  of  all  Operating  Systcas.  The  principal 
contribution  of  this  aodel  lies  in  the  structure  of  the  five 
other  levels. 

Logical  Pile  Systea 

To  present  the  goals  and  reguirenents  of  the  Logical 
Pile  Systea  in  a  brief  and  deaonstrative  aanner,  an  exaaple 
will  be  used.  The  reader  should  refer  to  Figure  4.2  for  the 
following  discussion.  It  is  ieportant  that  the  peculiarities 
of  the  exaaple,  such  as  the  choice  of  file  nates  (e.  g. 
"PILE6"  and  "DIF4"),  not  be  confused  with  the  general 
characteristics  of  the  Logical  File  Systea. 

In  Figure  4.2,  there  are  12  files  illustrated. 
Associated  with  each  file  is  an  identifier  of  the  fern 
"VOL  i  (3) ".  The  usage  of  this  identifier  will  not  be 
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Figure  4.2 

Exaaple  of  File  Directory  Structure  (to  IFS) 
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discussed  until  Inter,  in  the  neon while  notice  that  each 
file's  identifier  is  unique,  the  12  files  are  divided  into  2 
types,  directory  files  (i.«.  ton  (3),  voL2(3>,  vci3<2),  and 
VOL3  (5))  ,  and  data  files  (i.e.  V0L1(2),  V0i1(6),  toil  (hi, 

V011(M#  V012(4),  VOL2(2),  VOL3(4),  and  VOL3(3>>.  Th* 

distinction  between  directory  files  and  data  files  is  afilz  a 
aatter  of  usage,  the  Access  Rethods  jjj  operate  upon  a 
directory  file  in  the  sa ae  Banner  as  a  data  file, 
furthermore,  all  lewer  levels  (e.g.  Basic  Pilo  System)  treat 
ill  files  as  data  files,  this  factor  will  be  elaborated 
shortly. 

It  is  the  stated  function  of  the  Logical  File  systea  to 
■ap  a  file  nase  reference  into  a  unique  file  identifier. 
This  sapping  is  a  function  of  the  reguested  file  nase 
(symbolic  file  nase  path)  £ijd  a  starting  point  (base 
directory)  in  the  file  directory  structure.  In  Figure  4.2, 
three  exaiple  base  directories  are  illustrated  by 
associating  V0L1(3)  with  user  1,  VOL2(3)  with  user  2,  and 

VOL 3 (2)  with  user  3.  Therefore,  user  1  references  to  the 
file  nase  PIIE2  yields  the  file  V0L1(4). 

A  aore  coaplex  example  can  be  illustrated  by 
considering  the  file  VOL3(4).  user  3  can  refer  to  this  file 
under  the  naae  PILES ,  Alternatively,  it  can  be  referenced  by 
the  naae  DIR3.PILE7.  The  file  DIR3,  which  is  associated  with 
vol  3  (5 )  froa  user  3* s  base  directory,  is  interpreted  as  a 
lower  level  directory.  Then  froa  file  V0L3 (5) ,  the  file  nase 
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FI  187  is  capped  into  V0L3(«)  as  intend**.  The  file  V0L3(4) 
can  be  referenced  fro*  user  2*s  tag?  directory  as  DI83.FIi.8S 
or  DZ  83.DXF3,  FIIE7,  for  aiaaple.  Froa  nsec  1»e  base 
directory,  the  file  VOL3(4)  cab  fee  referenced  as  FXLS3, 
BIR2.BIB3.  FILES,  0182.  0183.  DIB  3*  PILE?,  or  even 

DIR 2. D IF  3. DIR 4 .DIE 3 .DIF  3 .FI IE 7. 

Two  iaportant  aide  affects  of  the  base  file  directory 
and  file  base  path  facilities  are  that  (1)  a  specific  file 
aay  be  referenced  by  aany  different  naies,  and  (2)  the  sane 
naae  aay  be  used  to  reference  aany  different  files. 

The  headings  VOLUME  "V0L1",  VOLUME  "V0L2*,  and  VOLUME 
«voi3«  are  intended  to  indicate  that  the  12  files  ace 
scattered  over  3  separately  detachable  volutes:  V0L1 
{containing  VOL1(2),  V0L1(3),  V0L1(U),  VOLI(S),  and 
VOL  1  (€) )  ,  VOL  2  (containing  VOL2{2),  VCL2(3),  and  VCL2(4)), 
and  VOL  3  (containing  VOL3(2),  VOL3(3),  VCL3(4),  and 
VOL  3(5))  .  If  volute  VOL2  vere  detached  froa  the  systee,  user 
1  could  still  reference  V0L1  (4)  as  FILB4  and  V0L3  (4)  as 
FILE3,  but  could  not  reference  V0L3  (4)  as  DIM.  DIS3.  FILES 
nor  V CL  1 ( 5)  as  EIB2.DIF3. IIR3.FILE6  since  the  path  vculd 
logically  reguiro  passing  through  volute  V0L2.  Furtheracre, 
user  3  is  allowed  to  erase  (i.e.  rteove  froa  file  systea 
structure)  the  file  VOL3(4)  under  the  naae  PILES,  assuring 
appropriate  protection  priviledges,  w^ethex  2£  fifit  voluae 
VOL  1  is  aounted  in  spito  of  user  I's  reference  to  file 
VOl3(4)  under  the  naae  PILF3 . 
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T be  Logical  Pile  Systes  could  be  extresely  ccsplex  if 
it  had  to  specif leal Ip  consider  the  physical  addresses  of 
volutes,  the  device  cha  tic  te  ri  t  ties,  and  the  location  of 
file  directories  ct  voluses,  in  addition  to  its  obvious 
requiretont  of  searching  file  directories.  These  problems 
are  eliainated  by  introducing  the  file  identifier  end  the 
interface  with  the  Sasic  Pile  systes. 

The  Basic  Pile  Systes  processes  reguests  that  specify  s 
file  in  teres  of  a  file  identifier  consisting  of  s  vcluse 
nase  and  index,  such  as  (VOL3,«),  rather  than  a  file  nsse.  A 
saeple  call  fros  the  Logical  Pile  Systes  to  the  Basic  Pile 
Systes,  in  PL/I-like  notation,  it: 

CALL  B PS_REAO  (VQLUHE,INDEX,COBE_ADDP,PILH_ADDB,CCUNT)  ; 
shore  VCLUNE  is  the  nase  of  the  voluse  containing  the  file, 
INDEX  is  the  corresponding  unique  index  of  the  file, 
CORE_ ADPH  is  the  sain  storage  address  into  which  data  is  to 
be  read,  PXLE_ADDP  is  the  file  virtual  sesory  address  fres 
which  the  data  is  to  be  read,  and  COUNT  is  the  nusber  cf 
bytes  to  be  transtitted.  Using  these  features,  the  heart  of 
the  loqical  Pile  Systes  (ignoring  opening  and  closing  files, 
file  access  protection,  illegal  file  oases,  etc.)  reduces  to 
the  tl/I-like  code  presented  in  Figure  4.3.  It  is  assused 
that  the  file  nase  has  been  broken  down  intc  an  array  of 
path  eleaent  naves  (e, g,  if  case  is  DIB2.DXB3.PILE8,  then 
PATH  (1)='DTB2*  ,  PATH  (2)  »  *DIR3\  PATH  (  3)  «  ‘PILES  • ,  and 


PATH  LFHGTH-3) 


that  BA SE.VOLUHE  and  £ASE_INDEX  initially 
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specif  j  the  { YOLOHI, IMDEX)  identifier  of  the  base  directory,, 
and  that  each  entry  in  a  file  directory  is  N  bytes  long  and 
formatted  as  indicated  in  the  FILE_fMTtY  declaration. 

For  efficiency,  the  nane3  of  all  files  that  are 
actively  in  use  {usually  a  seall  fraction  of  all  files  is 
the  systes)  are  kept  in  nain  storage  is  an  Active  Mane 
Directory  (AMD).  The  AND  is  searched  before  accessing  the 
file  directories!  cs  secondary  storage.  Entries  are  deleted 
froa  the  AMD  vhen  the  corresponding  file  is  “closed*  or 
“deleted*. 

Of  course,  the  handling  of  access  (protection)  rights, 
errors,  and  othar  responsibilities  will  sake  the  Logical 
rile  Systea  auch  aore  cov.plex,  but  it  is  isportant  tc  note 
that  the  design  and  iapleaentat ion  of  the  Logical  rile 
Syat«a  escapes  all  physical  file  organisation  and  device 
characteristic  considerations  and  coaplexities. 
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DEC!  ABE  1  FI  LEGIST  BE* 

2  FILENAME  CHARACTER  (8), 
2  VOL  ONE  CHARACTER  (8), 
2  INDEX  FIXED  BINARY , 


DO  I  *  1  TO  PATH_LENGTH; 

DO  J  *  0  BT  N  VHII.S  (FI  LE_E  NTRI .  FI  IE  B  AB  E  —  PATH  (I)); 
CALL  BF S_READ  (BA  SE_ VOLURb  ,8ASE_I  IDE! , FI LE_E NT  PI , J *H # N)  ; 
END; 

BASB..VOLUHB  *  PILE_EHTBT.  VOLtlHB; 

BASE_INDBX  =  FILB..BNTRY.INDE  X*» 

END; 


Figure  4.3 

Example  Procedure  to  Perfcta  Logical  File  Systea  Search 
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Basis  I ill  Slits* 

The  Basic  rile  System  aust  convert  the  file  identifier 
supplied  froa  the  Logical  rile  Systei  into  a  file  descriptor 
than  ear*  be  processed  by  the  File  organization  strategy 
nodule,  A  file  descriptor  contains  inforaation  such  as  the 
voluae  naae,  physical  location  of  the  file  on  the  volute, 
and  the  length  cf  the  file.  Every  file  aust  have  an 
associated  file  descriptor,  but  since  the  nuaber  of  passive 
files  (i.e.  not  actively  in  use)  tight  be  very  large,  the 
file  descriptors  are  raintained  on  secondary  storage  until 
needed  (i.e.  file  is  “opened") .  In  organizing  the  secondary 
storage  Maintenance  of  the  file  descriptors  there  are 
several  important  considerations: 

1.  There  aust  be  a  unique  file  descriptor  fex  each 
file  regardless  cf  ho*  often  the  file  appears  in 
file  directories  or  shat  symbolic  naaes  are  used. 
This  is  required  to  Maintain  consistent 
interpretation  of  a  file's  status. 

2.  The  file  descriptor  inforaation  for  a  file  aust 

reside  on  the  sane  voluae  as  the  file.  This  is 
reasonable  since  if  e i the r  the  file  or  its 

descriptor  is  not  accessable  at  soae  tiae  by  the 
systea  (i.e.  unmounted)  the  file  cannot  be  used, 
this  possibility  is  Minimized  by  placing  tbea  cn  the 


saae  volute 
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3.  In  the  .h»g  nannec  that  the  Logical  File  Syste* 
was  siiplified  by  using  the  facilities  of  the  lower 
hierarchical  level,  the  file  descriptors  should  be 
Maintained  in  a  Manner  that  allows  the  Pile 
Organization  Strategy  nodule  to  process  thea  as 
noraal  files. 

These  probless  are  salved  by  the  use  of  the  Voluse  file 
Descriptor  Directory  (VFDDJ .  there  is  a  single  VPDD  for  each 
voluse,  it  contains  the  file  descriptors  for  all  files 
residing  on  the  voluae.  The  file  descriptors  are  cf  fixed 
length  and  are  located  within  the  VPDD  positionally 
according  to  the  corresponding  file  identifier's  index.  In 
order  to  exploit  the  facilities  provided  by  the  Pile 
Organization  Strategy  nodule,  the  VPDD  can  be  processed  by 
the  lower  levels  as  a  nornal  file.  It  is  assigned  an  unigue 
file  identifier  consisting  of  the  volune  naae  and  an  index 
of  t,  in  fact  the  file  descriptor  for  a  VFDD  is  stored  (when 
not  in  use)  as  its  own  first  entry.  Figure  4.4  presents 
diagraanatically  the  logical  file  structure  cf  Figure  4.2 
with  the  added  detail  of  the  Voluae  File  Descriptor 
Directories  and  File  Directory  fornats. 

For  efficiency,  the  descriptor's  of  all  files  that  are 
active  ly  in  use  are  stored  in  an  active  File  Directory 
(AFD).  The  AFD  is  searched  before  accessing  the  Voluae  File 
Descriptor  Directory. 
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Figure  4 .4 

Exaiple  of  File  Directory  Structure  (to  BFS) 
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The  Pile  Organization  Strategy  Nodule  processes 
requests  that  specify  a  file  in  teras  of  a  file  descriptor 
(the  entry  extracted  froe  the  VPDD)  rather  than  a  file  eaae 
or  file  identifier.  A  saaple  call  froa  the  Basic  Pile  systes 
to  the  Pile  organization  strategy  Jlodule,  is  PL/X-like 
notation,  is: 

C A 1 1  FCSH_READ  (DESCRIPTOR,  CORE.AEDR,  PILE_ADDR, COUNT)  ; 
where  COHE_A  DDR ,  PILE  JIDDR,  and  COUNT  have  the  saae 

interpretation  as  discussed  above. 

The  priaary  function  of  the  Basic  Pile  Systea  reduces 
to  the  single  request: 

CALL  POSM_REAC(yPDD_DESCRIPTOR, DESCRIPTOR,*!*  (INCIX-1)  ,  E)  { 
Where  VFDD_DESCPIFTOR  is  the  descriptor  of  the  VPDD 

associated  with  the  voluae  naae  supplied  by  the  Logical  File 
Systea  as  part  cf  the  file  identifier,  INDEX  is  frca  the 
specified  file  identifier,  .1  is  the  standard  length  cf  a 
VPDD  entry,  and  DESCRIPTOR  is  the  desired  file  descriptor. 

The  Basic  Pile  Systea  perforas  several  other  tasks, 
such  as  protection  validation  and  aaiutenance  of  the 
core-resident  Active  Pile  Directory  that  enables  efficient 
association  between  a  file's  identifier  and  descriptor  for 
files  that  are  in  use  (i.e.  "open").  But,  as  in  the  Logical 

Pile  Systea,  the  doaain  of  the  Basic  Pile  Systex  is 

sufficiently  snail  and  narrow  *hat  it  retains  a  conceptually 
siaple  level  in  the  hierarchy. 
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rile  omfilmigB.  Jitiitai  gg^alss  • 

The  Logical  File  Systes  and  Basic  File  Systea  ace,  tc  a 
great  extent,  application  and  device  independent.  The  Pile 
Organization  Strategy  Nodules  are  usually  the  aost  critical 
area  of  the  file  systea  in  terns  of  overall  perfcraance,  for 
this  reasou  it  is  expected  that  sore  than  one  strategy  aay 
be  used  in  a  large  systea.  Only  one  strategy  will  be 
discussed  in  this  section,  the  reader  say  refer  to  the 
papers  listed  in  the  Bef  erences<Corb  62><Nad  68bXSalt 
65><scie  68>  for  other  possible  alternatives. 

The  FOSN  aust  nap  the  logical  file  address  ontc  a 
physical  record  address  or  hidden  buffer  based  upen  the 
supplied  file  descriptor  information.  In  the  sisplest  case, 
the  sapping  could  be  performed  by  including  a  two-part  table 
in  the  file  descriptor.  The  first  part  of  each  entry  would 
indicate  a  contiguous  range  of  virtual  file  addresses,  the 
second  part  of  each  entry  would  designate  the  corresponding 
physical  record  address.  It  has  been  assuaed,  however,  that 
all  file  descriptors  have  a  specific  length,  whereas  the 
napping  table  is  a  function  of  the  file's  length  and  is 
potentially  guite  large.  Therefore,  it  is  not  feasible  to 
include  the  entire  sapping  table  as  part  of  the  file 
descriptor.  One  of  the  aost  powerful  file  organization 
strategies  utilizes  file  aaps.  Figure  4.5  illustrates  such 
an  arrangeaent. 
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Figure  4,5 

Exaaple  of  Pile  Organization  Strategy 
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In  this  esaaple  it  is  assuaed  that  each  file  is  divided 
into  1000  byte  physical  records.  A  file  can  be  is  cue  of 
several  states  depending  upon  its  current  length.  Sf  the 
file's  length  is  in  the  range  1  to  999  bytes*  the  file 
descriptor  contains  the  address  of  the  corresponding 
physical  record.  If  the  file  is  fcetveen  1000  and  9$9,999 
bytes  long*  the  file  descriptor  specifies  the  address  cf  a 
file  sap  located  on  secondary  storage.  Each  entry  of  the 
file  sap  (assuaed  to  require  2  bytes)  designates  the 
physical  address  of  a  block  of  the  file  (blocks  are  ordered 
by  virtual  file  addresses:  0-999,  1000-1999,  2000-2999* 

etc.).  Purthersore*  for  files  greater  than  500,000  bytes, 
but  less  than  250,000,000  bytes,  there  are  2  levels  cf  file 
saps  as  illustrated. 

This  strategy  has  several  advantages.  Onder  the  verst 
conditions  of  randot  access  file  processing  only  fres  one  to 
three  X/0  operations  need  to  be  perforaed.  By  utilising 
several  hidden  buffers  for  blocks  of  the  file  as  veil  as 
file  saps,  the  nusber  of  X/O  operations  required  for  file 
accesses  can  be  drastically  reduced. 
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The  function  of  Allocation  and  deallocation  of  blocks 
in  vulvas  several  separate  factors.  Bator#  describing  the 
i  »pl*»aon  tation  of  the  aecbanisas,  it  is  vise  to  ceviev  the 
desired  characteristics: 

1.  A  file  is  allowed  to  grow  in  size,  the  FOSH  will 
request  additional  blocks  froa  the  ASK  for  the 
data  portions  of  a  file  or  its  index  tables,  as 
needed . 

2.  Coaaon  direct  access  devices  contain  fro«  8000  to 
32C00  separately  allocatable  blocks,  thus  it  is 
not  feasible  to  store  all  allocation  infcrvaticn 
in  aain  storage, 

3.  since  tec  independent  processors  aay  be  writing 
new  files  on  the  saee  vcluae  at  the  saee  tiae,  it 
is  necessary  to  provide  interlocks  such  that  they 
do  not  accidently  allocate  the  saae  block  to  sore 
than  one  file,  yet  not  require  one  processor  to 
wait  until  the  other  processor  finishes. 

These  probless  can  be  solved  by  use  of  a  special  Voluae 
Allocation  Table  (VAT)  on  each  voluac.  In  this  schece,  a 
voluie  »ust  be  subdivided  into  arbitrary  contiguous  areas. 
For  direct  access  devices  with  aovable  read/write  beads, 
each  discrete  position  (kncwn  as  a  "cylinder")  covers  an 
area  of  about  <JC  tc  160  blocks.  A  cylinder  is  a  reasonable 
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u  n  it  of  subdivision.  roc  each  cylindtr  oa  the  vcluae,  there 
is  a  cor responding  entry  in  the  VAT.  Each  entry  contains  a 
"bit  amp"  that  indicates  sbich  blocks  oa  that  cylinder  have 
not  teen  allocated.  Por  example,  if  a  cylinder  consists  of 
40  blocks,  the  bit  sap  in  the  corresponding  VAT  entry  would 
be  40  bits  long.  If  the  first  bit  is  a  "0",  the  first  block 
has  not  been  allocated;  if  the  bit  is  a  "1",  the  block  has 
already  been  allocated.  Likewise  for  the  second,  third,  and 
reaaining  bits. 

When  the  rose  first  requests  allocation  of  a  blcck  cn  a 
volume,  the  ASM  selects  a  cylinder  and  requests  that  the  DSN 
read  the  corresponding  VAT  entry  into  aain  storage.  An 
available  block,  indicated  by  a  "0"  bit,  is  located  and  then 
narked  as  allocated.  As  long  as  the  volune  resains  in  use, 
the  VAT  entry  will  be  kept  in  sain  storage  and  blocks  vill 
be  allocated  on  that  cylinder.  Vhen  all  the  blocks  on  that 
cylinder  have  been  allocated,  the  updated  VAT  entry  is 
written  out  and  a  new  cylinder  selected.  Mith  this  technique 
the  aeount  of  sain  storage  required  for  allocation 
information  is  kept  to  a  siniaua  (about  40  to  160  bits  per 
volume),  at  the  sane  tiae  the  nuaber  of  extra  I/O  operations 
is  minimized  (about  one  per  40  to  160  blocks  of  allocation) . 

The  problem  cf  interlocking  the  independent  processors 
still  remains.  As  long  as  the  processors  are  allocating 
blocks  on  different  cylinders  using  separate  VAT  entries, 
they  may  both  proceed  uninterrupted.  This  condition  can  be 
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Accomplished  by  utilizing  a  hardware  fmature  kneve  as  "keyed 
records"  available  on  several  computers  including  the  IBM 
Syatte/36?.  Each  c £  the  V AT  entries  is  a  separate  rocord 
consisting  of  a  physical  key  area  and  a  data  area.  The  data 
urea  contains  the  allocation  information  described  above. 
The  key  area  is  divided  into  tvo  parts:  the  identification 
nuaber  of  the  processor  currently  allocating  blocks  on  that 
cylinder  and  an  indication  if  all  blocks  on  that  cylinder 
have  been  allocated.  A  VAT  entry  vit h  a  key  of  all  zeroes 
would  identify  a  cylinder  that  was  not  currently  in  use  and 
had  blocks  available  for  allocation. 

There  ace  I/O  instructions  that  can  be  used  by  the  DSN 
that  will  automatically  search  for  a  record  with  a  specified 
key,  such  as  zero.  Since  the  device  controller  will  not 
switch  processors  in  the  midst  of  m  continuous  stream  of  I/O 
operations  from  a  processor  (i.e.  "chained  I/C  commands") , 
it  is  possible  to  generate  an  uninterruptible  sequence  of 
I/O  commands  that  will  (1)  find  an  available  cylinder  by 
searching  the  VAT  for  a  entry  with  a  key  of  zero  and  (2) 
change  the  key  to  indicate  the  cylinder  is  in  use.  This  thus 
solves  the  aul ti -processor  allocation  interlock  problem. 
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Tb*  Device  strategy  nodules  convert  ^logical  I/O 
requests*  free  the  file  Organization  Strategy  nodules  and 
Allocation  Strategy  nodules  into  actual  coaputer  I/O  coaaand 
sequences  that  are  forwarded  to  the  Input/Output  Control 
Systea  for  execution. 

when  a  request  to  transfer  a  large  portion  of  a  file 
(10,000  bytes  for  exaaple)  is  issued,  it  is  unlikely  that  a 
significant  aaount  cf  the  needed  blocks  are  in  hidden 
buffers.  Xt  will#  therefore,  be  necessary  to  request  I/O 
transfer  for  several  blocks  (e.g.  about  10  blocks  if  each 
block  1000  bytes  long) .  The  FOSH  will  generate  logical  I/O 
requests  of  the  fors:  "read  block  227  into  location  12930, 
read  block  211  into  location  13930,  etc."  The  DSH  aust 
consider  the  physical  characteristics  of  the  device  such  as 
rotational  delay  and  "seek"  position  for  aovable  beads.  It 
then  decides  upon  an  optiaal  sequence  to  read  the  blocks  and 
generate  the  necessary  physical  I/O  coaaand  sequence 
including  positioning  coaiands.  The  Xnput/Output  Ccntrol 
Systea  actually  issues  the  physical  I/O  request,  error 
retry,  and  ether  housekeeping  as  discussed  earlier.  The 
detailed  strategy  for  choosing  the  optinal  I/C  sequence  is, 
of  course,  very  device  dependent  and  will  not  be  elaborated 
here . 
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The  preceeding  sections  have  highlighted  the  framework 
of  a  file  system.  There  are,  of  course,  many  other  important 
decisions  to  be  fade  ia  such  a  systea,  sueh  as  the  format 
and  organization  of  tables,  error  conditions<Lock  68>, 
measurement  and  accounting  mechanisms,  etc.  Cne  of  the 
subtle  points  will  be  discussed  in  this  section. 

The  Basic  Pile  Systea  is  intended  to  deal  with  files 
reoresented  by  unique  identifiers.  In  the  specific  system 
presented,  the  identifier  is  designated  as  the  tuple, 
Cvolume,  index  in  VFDD>.  This  representation  resulted  in  a 
very  efficient  mechanism  for  accessing  a  file's  descriptor 
that  avoided  much  of  the  tiae-consua ing  table  lcck-up. 
Unfortunately,  this  representation  is  not  temporally  unique. 
It  has  teen  assuaed  that  when  a  file  is  deleted,  the  VPDD 
index  position  used  for  that  file's  descriptor  is  available 
for  use  by  new  files  that  may  be  created.  This  would  not  be 
a  problem  if  all  instances  of  the  deleted  file's  identifier 
were  removed  from  the  system  at  the  same  time,  but  there  may 
be  more  than  one  path  to  the  file  due  to  links  from  ether 
symbolic  file  directories.  The  strategy  used  by  the  Basic 
File  system  did  net  provide  any  convenient  means  tc  lccate 
all  references  (i.e.  links)  to  a  specific  file.  Furthermore, 
oven  if  such  a  mechanism  existed,  it  would  not  solve  the 
crobler  since  the  reference  may  exist  in  a  file  directory 
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that  is  located  on  a  voluae  that  is  not  physically  acunted 
or  accessable  by  the  syrtea  at  the  tine  of  deletion. 
Therefore,  in  such  an  environaent,  it  is  possible  to  have 
links  in  directories  that  identify  files  that  have  been 
deleted.  The  danger  exists  that  the  following  sequence  of 
events  nay  occur:  (1)  a  file  is  created  and  assigned 
identifier,  <ALPHA,5>,  (2)  a  link  is  Bade  to  that  file,  (3) 
the  file  is  deleted  by  its  creator,  (4)  a  new  file  is 
created  and  coincidently  assigned  the  identifier  <ALPHA,5>, 
and  (5)  the  link  previously  created  is  used  not  realizing 
that  the  intended  file  has  been  deleted  and  replaced  by  scue 
other  arbitrary  file! 

Fortunately,  this  diletta  is  not  irrevocable,  there  is 
a  aultitude  of  solutions.  Two  simple  variations  would  be  (1) 
never  reuse  VPDD  entries  but  allow  the  file  tc  continually 
grow  but  becone  "sparse"  or  (2)  maintain  count  of  the  nuiber 
of  links  to  a  file  and  reuse  the  VFDD  entry  only  when  all 
links  have  been  reaoved,  A  better  solution  can  be  formulated 
by  attacking  the  original  goal  of  generating  truly  unique 
file  identifiers.  The  Hultics  Operating  Syste»  has  sisilar 
requirements,  it  fcrBS  unigue  identifiers  by  concatenating 
the  central  processor's  unique  serial  number  with  the 
chronolog  clock  tiae  with  accuracy  in  the  range  of 
aicrcseconds ,  A  much  simpler  scheme  can  be  incorporated  into 
the  file  system  by  associating  a  separate  counter  with  each 
volume.  Whenever  a  new  file  is  created  on  a  volume  and 
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assigned  a  VPDD  entry,  the  value  of  the  corresponding 
counter  is  incremented  by  one.  Por  the  purpose  cf  the  file 
system,  the  tuple,  <voluae,  counter  vaiue>,  is  a  unique 
identification  of  a  file. 

The  counter  value,  which  sonotcaically  increases, 
cannot  be  efficiently  used  as  a  direct  index  into  a  finite 
size  file  descriptor  directory,  k  minor  aodification  to  the 
Basic  File  Systea  design  can  incorporate  the  ideas  of  the 
above  discussion.  The  file  identifier  can  be  constructed 
from  the  triple,  Cvolune,  VPDD  index,  counter  value>.  In 
this  context  the  counter  value  will  be  called  a  •'key",  since 
its  sole  purpose  is  to  verify  that  the  accessed  VPDD  entry 
is  correct  by  attempting  to  "unlock"  the  entry  (i.e. 
comparing  the  key  fro*  the  VPEE  entry  with  the  key  froa  the 
symbolic  file  directory  which  was  copied  froa  the  VPDD  when 
the  link  was  initially  established). 

The  above  prcbleas  are  typical  of  the  factors  that  must 
be  considered  by  file  systea  designers.  The  general  file 
system  model  vill  very  seldom  be  a  coaplete  description  of  a 
specific  implementation  and  it  certainly  will  net  replace 
the  need  for  systems  analysts,  but  it  can  save  aany  Berths 
of  the  initial  design! 
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CONCLUCING  COBHINTS 

Tc  a  large  extent  file  systems  ace  currently  developed 
and  implemented  in  cuch  the  saae  aannec  as  early  "horse-less 
carriages",  that  is,  each  totally  unique  and  "hand-made" 
rather  than  "aass  produced".  Compilers,  such  as  FOBTFAN, 
were  once  developed  in  this  priaative  Banner;  but  due  to 
careful  analysis  of  operation  (e.g.  ,  lexical,  syntax,  and 
semantic  analysis,  etc.),  co»pi lers  are  sufficiently  veil 
understood  that  certain  software  coapanies  actually  ctfer 
"do-it-yourself  FORTRAN  kits".  Since  nodern  file  systexs 
often  outweigh  all  ether  operating  system  components  such  as 
compilers,  loaders,  and  supervisors,  in  terns  of  programmer 
effort  and  number  of  instructions,  it  is  important  that  a 
generally  applicable  methodology  be  found  for  file  system 
development . 

This  paper  presents  a  modular  approach  to  the  design  of 
general  purpose  file  systems.  Its  scope  is  broad  encugh  tc 
encompass  most  present  file  systems  of  advanced  design  and 
file  systems  presently  planned,  yet  basic  enough  to  be 
applicable  tc  lore  modest  file  systems. 

The  file  system  strategy  presented  is  intended  to  serve 
tvo  purposes:  (1)  to  assist  in  the  design  of  nev  file 
systems  and  (2)  to  provide  a  structure  by  vhich  existing 
file  systems  may  be  analyzed  and  cciparcd. 
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